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ABSTRACT 

This  report  deals  with  the  problem  of  detecting  a  known  signal  in 
non-Gauss ian  or  dependent  noise.  Although  likelihood  ratio  (LR)  detec¬ 
tors  are  discussed,  primary  attention  is  paid  to  asymptotic  detector  per¬ 
formance,  and  therefore  to  maximum  efficacy  or  locally  optimal  (LO) 
detectors.  In  general,  the  detectors  considered  consist  of  a  nonlinearity 
followed  by  a  filter  and  a  threshold  comparator. 

The  asymptotic  performance  of  three  common  suboptimal  detectors 
is  considered  for  several  families  of  noise  densities  as  the  input  signal- 
to-noise  ratio  (SNR)  varies.  Contours  of  equal  detector  performance  are 

f 

plotted  allowing  the  relative  utility  of  the  detectors  to  be  assessed. 

A  method  of  designing  suboptimal  detector  nonlinearities  is 
presented.  A  suboptimal  nonlinearity  is  chosen,  and  the  family  of  densi¬ 
ties  for  which  it  is  locally  optimal  is  found.  A  member  of  this  family  is 
then  fitted  to  the  observed  noise,  and  the  corresponding  detector  is  used. 
When  the  nonlinearity  is  a  rational  function,  the  Pearson  family  of  densi¬ 
ties  results.  Not  only  does  this  family  contain  many  common  univariate 
densities,  but  a  member  density  can  be  fitted  to  an  observed  noise  using 
only  the  first  four  sample  moments.  Other  possible  nonlinearities  are 


considered,  including  possible  multivariate  extensions. 

A  ten  minute  sample  of  Arctic  under-ice  ambient  noise  is  subjected 
to  a  moment  analysis  as  suggested  above.  The  noise  is  found  to  be  non¬ 
stationary  and  largely  Gaussian  or  nearly  Gaussian  with  sporadic  bursts 
of  non-Gaussian  noise. 

The  LR  and  LO  detectors  for  several  classes  of  multivariate  densities 
are  given.  These  classes  include  closed  form,  differential  equation,  spher¬ 
ically  symmetric,  series  expansion,  transformation  and  moving  average 
models.  The  transformation  model,  (noise  generated  by  a  memoryless, 
nonlinear  transformation  of  a  correlated,  Gaussian  source)  is  discussed  in 
some  detail.  The  performance  and  practical  aspects  of  obtaining  the  sub¬ 
systems  of  the  transformation  LO  detector  are  considered.  Applicability 
of  the  noise  models  and  tractability  of  the  resulting  detectors  are  dis¬ 
cussed. 

The  problem  of  finding  the  minimum  length  matched  filter  in 
discrete-time  to  achieve  a  desired  level  of  performance  (SNR)  is  con¬ 
sidered  when  there  is  some  freedom  in  choosing  a  signal  shape.  Exact 
and  approximate  upper  and  lower  bounds  on  the  SNR  are  given,  and  the 
problems  of  optimal  and  suboptimal  signal  design  are  discussed. 
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CHAPTER  1  -  INTRODUCTION 


THE  DETECTION  PROBIZM 

The  detection  problem  considered  here  is  basically  a  simple  one.  A 
detector  must  determine  whether  a  signal  or,  to  generalize  slightly, 
which  of  a  number  of  possible  signals,  is  present  at  its  input.  Were  there 
no  noise  contaminating  the  received  signal  the  problem  would  be  moot. 
However,  with  the  presence  of  noise,  often  of  an  unknown  or  incompletely 
known  character,  the  choice  of  detector  structure  can  be  quite  difficult. 
This  problem  occurs  in  many  areas  unrelated  to  communications,  but 
unlike  many  of  those  areas,  decisions  frequently  must  be  made  in  "real 
time"  and  with  guaranteed  bounds  on  the  probability  of  error. 

While  there  are  many  possible  measures  of  detector  performance, 
asymptotic  results  are  used  throughout  this  dissertation,  often  without 
much  background  or  justification.  While  this  makes  each  chapter  more 
readable  (and  less  repetitious),  a  fuller  development  of  asymptotic 
theory  can  increase  understanding.  A  more  complete  discussion  of 
asymptotic  theory  can  be  found  several  recent  books.  Pitman’s  [1] 
development  is  fairly  rigorous  and  complete.  Of  particular  interest  in  the 
framework  of  this  dissertation,  Pitman  derives  the  efficacy  as  a  measure 
of  asymptotic  detector  performance,  and  discusses  some  of  the  proper¬ 
ties  of  both  efficacy  and  the  locally  optimal  detector.  Huber  [2]  uses 
asymptotic  results  to  develop  his  theory  of  robustness  for  both  detectors 
and  estimators.  A  more  general  background,  clearly  placing  asymptotic 
results  within  the  broader  area  of  statistical  inference  theory  is  given  by 
Bikel  and  Doksum  [3],  and  by  Helstrom  [4]  who  approaches  the  problem 


more  from  an  engineering  point  of  view. 

In  the  sections  which  follow,  some  of  the  basic  results  of  asymptotic 
theory  are  presented,  both  to  make  the  notation  clear,  and  to  give  some 
background  for  the  following  chapters.  Fisher’s  Information  is  derived, 
and  shown  to  be  a  measure  of  the  sensitivity  of  a  density  to  differential 
changes  in  a  parameter  D.  The  efficacy  is  then  shown  to  be  related  to 
Fisher's  Information  through  the  Information  Inequality.  And  finally  the 
efficacy  is  shown  to  be  an  indicator  of  the  asymptotic  behavior  of  a  detec¬ 
tor.  The  comments  made  here  are  not  intended  to  form  a  rigorous 
development  of  asymptotic  theory;  for  that  the  reader  is  referred  to  any 
of  the  References  listed  above. 

Detection  and  estimation  problems  are  generally  concerned  with  a 
set  of  probability  measures  \P+  e  0}  indexed  by  a  real  parameter  (or 
parameters)  -9  contained  in  an  interval  0.  This  set  has  a  corresponding 
set  of  densities  e  0J  and  an  associated  o-finite  measure  /x.  In  the 

detection  problem  considered  in  this  dissertation  is  generally  taken  to 
be  the  signal  level,  and  the  detector  is  asked  to  decide  whether  iJ  =  t>0  or 

>t>0- 

1.  Fisher's  Information 

The  sensitivity  of  a  detector  or  estimator  to  changes  in  the  parame¬ 
ter  if  is  frequently  of  interest.  This  would  give  an  indication  of  detector 
or  estimator  performance  for  small  signal  level,  i?  *  tf0.  As  shown  below, 
Fisher’s  Information  can  be  viewed  as  the  sensitivity  of  a  density  to 
differential  changes  in 


As  a  measure  of  how  two  densities  differ  at  a  point  x,  consider  the 
likelihood  ratio  (LR): 

A(*)  =  /*(*)/ /*,(*) 

or  equivalently  the  log-likelihood  ratio: 

Mx)  =  ln/«(z)  -  ln/^z) 

Both  the  LR  and  log-LR  detectors  are  well  known  [1-4]  to  be  Neyman- 
Pearson  optimal  for  deciding  between  /*  and  /*0  based  on  an  observation 
z.  Because  the  LR  and  log-LR  are  optimal  statistics  for  distinguishing 
between  two  densities,  either  can  also  be  viewed  as  a  measure  of  the  dis- 
tinguishability  of  the  two  densities  at  a  point  z .  The  derivation  which  fol¬ 
lows  would  be  essentially  the  same,  and  would,  in  fact,  yield  identical 
results  whether  the  LR  or  log-LR  is  used  as  a  measure  of  distinguishabil- 
ity.  However,  the  log-LR  is  used  since  it  has  several  appealing  charac¬ 
teristics.  Without  sacrificing  the  optimality  of  the  LR,  the  log-LR  equals 
zero  when  two  densities  are  indistinguishable  at  z,  that  is  when 
/«<*) = /«.<*  ).  It  also  has  an  appealing  symmetry  since 


In  (/*//*,)  =  "In  (/*//«) 

The  sensitivity  of  / «  to  a  change  in  can  be  written  as  the  ratio  of 
the  distinguishability  (Le.  log-LR)  to  the  change  in 

S(x)  =  *(«)  -  to /«(*)- *n /«,(«) 

T?- T?0  (if— ' do) 

As  iJ  -*  i?o  the  result  is  the  rate  of  discrimination: 

x  to/ *(z)  -  to/*  (z) 

lun  S(x)  =  lxm - — 2 - 

Assuming  that  i?  is  an  element  of  a  real  interval,  and  that  the  1?  derivative 


-4  - 


of  ft  exists  at  i?o.  then 


UmS(x)=Um^rln/4(*)  = 


/* „(*) 


where  denotes  the  iJ  derivative  of  at  =  tJ0.  Denoting  the  log- 


likelihood  function  at  i?0  as 


Wx>  =  In /*,(*) 

Then  the  rate  of  dis crimination  is  the  i?  derivative  of  the  log-likelihood 
function: 


KmS(x)  -  I*0'(x)  =  /*„’(*)/ /*„(x)  (1) 

The  rate  of  discrimination  L^{x)  is  a  random  variable  with  expected 
value  zero: 

W  =  //«;(*)**  =  o  (2) 

As  a  measure  of  overall  discrimination,  consider  the  variance  of  I*0’: 

V*M'  =  (3) 

which  is  seen  to  equal  7(i>o)»  Fisher’s  information  at  fl0. 

Fisher’s  information  can  be  interpreted  to  measure  the  overall  sensi¬ 
tivity  of  a  density  to  differential  changes  in  a  parameter  i?  at  t?0-  In 
another  light,  it  can  be  viewed  as  the  distinguishability  of  f  +  and  as 
-*  T>o- 

2.  Efficacy  and  Information  Inequality 

Fisher's  Information  can  be  shown  to  have  a  corresponding  measure 
of  asymptotic  detector  (or  estimator)  performance,  the  efficacy. 
Further,  the  Information  Inequality  (or  Cramer-Rao  bound)  states  that 
Fisher’s  Information  is  an  upper  bound  on  efficacy. 


Let  T(x)  a  statistic  of  the  observation  x  with  density  /#(z)  with 
expected  value  given  by 

E*T  =  /  T(x)f *(*)dM 

The  density  is  assumed  regular  in  Pitman’s  sense  [1],  allowing  the  t? 
derivative  of  E+T  to  be  brought  inside  the  integral: 


=  £Y  = 


and  using  Eq  (1): 


£*T  =  fT(x)L^(x)/^x)dfj,  (4) 

Since  E4L4  =  0  from  Eq.  (2),  then  an  examination  of  Eq.  (4)  reveals  that 

E+T  is  the  covariance  of  T  and  L+.  Thus  by  the  correlation  inequality: 

(EJT)**  V+TV+LJ 

with  equality  if  and  only  if  T  -  k  V  where  k  is  a  real  constant.  From  Eq. 
(3),  V4L4  -  /(t>).  The  result  is  the  Information  Inequality: 


E+T2 

V*T 


*  1(4) 


(5) 


which  can  be  slightly  rearranged  to  yield  the  Cramer-Rao  lower  bound. 
The  quantity  on  the  left  side  of  the  inequality  is  defined  as  the  efficacy  of 
the  statistic  T{x)  at  and  is  denoted 


J(T)  =  Ej Tz/  V+  T 

It  follows  from  the  correlation  inequality  that  the  efficacy  is  maximized 
by  the  test  statistic  =  L+.  This  test  statistic  is  referred  to  as  the 
locally  optimal  test  statistic,  and  has  efficacy  equal  to  Fisher's  Informa¬ 
tion,  J(Tio)  =  7(iJ). 


3.  Asymptotic  Power  of  a  Detector 

The  efficacy  can  be  shown  to  measure  the  asymptotic  behavior  of  a 
detector  for  large  n  and  small  tJ.  A  general  but  loose  proof  of  this  is 
given  below.  This  can,  and  indeed  has  been  made  more  rigorous,  and  a 
more  detailed  development  is  given  by  Pitman  [l]. 

As  stated  above,  the  detection  problem  considered  consists  of  testing 
the  hypothesis  that  =  i>o  against  the  alternative  that  >  tf0>  where  t? 
and  U0  are  contained  in  the  real  interval  8.  The  detector  consists  of  a 
test  statistic  Tn  which  is  compared  to  a  threshold  tn  to  decide  for  the 
hypothesis  or  alternative.  Here  n  is  the  observation  size,  and  Tn  has 
expected  value  E*Tn  =  /z*  and  finite  variance  V^Tn  =  of.  Then  if  the  Cen¬ 
tral  Limit  Theorem  holds,  as  n -*«>,.  the  random  variable  (Tn  —  fj^)/ is 
asymptotically  normal  with  distribution  $  for  all  iS.  The  approximate 
level  an  of  Tn  can  thus  be  given,  for  large  n,  by: 


On  =  P*0[Tn  >  tnJ*  ~  *«)/  °*0] 

The  threshold  can  be  found  in  terms  of  a* : 

*n  *  ^#0  -  o+p'Han) 

In  a  similar  fashion,  the  approximate  power  of  a  test  is  given  by 

0n  =P*[Tn>tn]**[(^-tn)/04] 

Combining  Eqs.  (6)  and  (7)  yields  the  approximate  power  function: 


(6) 


(?) 


0(a)  =  $ 


2S4-W  - 


Using  the  mean  value  theorem,  we  can  write 


where  prime  indicates  differentiation  with  respect  to  and  tJ*  lies 


between  and  i?o-  The  power  function  can  be  written  as 


/5(a)  =  $ 

For  convenience,  define  d  =  Then  for  large  n,  d  dominates  the 

expression  in  brackets,  and  the  power  function  is  a  monotone  increasing 
function  of  d.  Since  d  ^  0,  maximizing  d2  is  equivalent  to  maximizing  the 
power  function.  For  « i>0,  then  tJ*  w  i?0.  and  d2  is  very  nearly  equal  to 
the  efficacy  J(Tn): 


a*  a* 


(*o2 


“  W«t,-  a| 


W)2 


Thus  for  large  observation  size  n,  and  iJ  « iS0,  maximizing  the  efficacy  is 


equivalent  to  maximizing  the  power  function.  It  can  also  be  shown  that 
maximizing  the  efficacy  is  equivalent  to  maximizing  the  slope  of  the 


power  function  at  =  i?0. 


DISSERTATION  OUTLINE 

This  dissertation  deals  with  the  problem  of  detecting  a  known  signal 

f 

in  additive  noise.  Several  of  the  traditional  assumptions  are  relaxed;  the 
noise  is  assumed  to  be  non-Gaussian  and/or  dependent.  In  addition,  the 
statistical  mode.)  is  often  assumed  to  be  incompletely  known. 

In  Chapter  2,  the  performance  of  several  common  detectors  is  con¬ 
sidered  for  several  families  of  noise  densities  as  the  input  signal-to-noise 
ratio  (SNR)  varies.  The  detector  consists  of  a  nonlinearity  followed  by  a 
filter  and  a  threshold  comparator.  Three  commonly  used  nonlinearities 
are  investigated:  the  linear  amplifier,  the  sign  detector  and  the  amplifier 
limiter.  Contours  of  equal  detector  performance  are  plotted  allowing  the 
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relative  utility  of  the  three  detectors  to  be  assessed  for  the  noise  models 
considered.  The  investigation  was  motivated  by  the  observation  that  the 
locally  optima]  (or  small  signal)  detector  is  designed  for  zero  input  SNR 
and  can  perform  rather  poorly  as  the  input  SNR  increases.  While  this 
observation  is  bora  out,  some  additional  conclusions  can  be  drawn.  In 
particular,  the  performance  of  the  sign  detector  falls  off  rather  quickly 
with  increases  in  input  SNR. 

It  is  well  known  that  the  locally  optimal  detector  for  known  signal  in 
noise  consists  of  a  nonlinearity  followed  by  a  matched  filter.  In  Chapter 
3,  suboptimal  nonlinearities  are  investigated.  It  is  assumed  that  either 
the  locally  optimal  nonlinearity  is  too  complex  to  use  or  that  the  noise 
density  is  not  known  precisely.  A  suboptimal  nonlinearity  can  be  chosen, 
and  the  family  of  densities  for  which  it  is  optimal  is  found.  A  member  of 
this  family  is  then  fitted  to  the  observed  noise,  and  the  corresponding 
detector  is  used.  When  a  rational  function  is  chosen  for  the  nonlinearity, 
the  Pearson  family  is  the  set  of  solution  densities.  This  is  not  only  a  gen¬ 
eral  family  which  contains  many  common  univariate  densities,  but  for 
nearly  Gaussian  noise  the  method  of  moments  can  be  used  efficiently  to 
fit  a  member  density  to  the  noise.  The  coefficients  of  a  ZNL  are 
estimated  for  several  (non-Pearson)  densities  using  the  first  four  noise 
moments.  Performance  of  the  suboptimal  detectors  is  investigated. 
Other  possible  nonlinearities  are  considered,  including  multivariate 
extensions. 

In  Chapter  4,  10  minutes  of  Arctic  under-ice  ambient  noise  is  sub¬ 
jected  to  a  moment  analysis  similar  to  that  suggested  in  chapter  3.  The 
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noise  is  found  to  be  nonstationary  and  largely  Gaussian  or  nearly  Gaus¬ 
sian  with  sporadic  bursts  of  non-Gaussian  noise.  Unfortunately, 
insufficient  data  is  available  for  more  extensive  moments  analysis,  but 
the  tentative  conclusions  drawn  about  the  nature  of  the  noise  are 
interesting. 

Chapters  5  and  6  (excluding  Appendix  B  of  Chapter  5)  were  coau¬ 
thored  with  P.F.  Swaszek  and  also  appear  in  his  dissertation  [5]. 

In  solving  problems  in  detection,  it  is  often  assumed  that  the  under¬ 
lying  statistical  description  is  independent  or  Gaussian.  Not  making 
these  assumptions  leads  to  difficulties  in  detector  design  due  to  problems 
visually  encountered  in  specifying  multivariate  noise  statistics.  In 
Chapters  5  and  6,  several  characterizations  of  multivariate  densities  are 
considered  within  a  detection  framework.  The  discussions  include 
specific  examples  and  also  some  general  methods  of  density  generation. 

The  particular  detection  problem  considered  in  Chapter  5  is  that  of  a 
known  signal  with  a  vanishingly  small  amplitude  in  additive  noise. 
Efficacy  is  employed  as  a  criterion  of  detector  performance  and  the  max¬ 
imum  efficacy  (locally  optimal)  detector  is  discussed.  The  class  of  mul¬ 
tivariate  densities  generated  by  a  memoryless,  nonlinear  transformation 
of  a  correlated,  Gaussian  source  is  discussed  in  some  detail.  A  member 
of  this  class  has  the  advantage  of  being  completely  characterized  by  its 
marginal  density  and  its  covariance  matrix.  The  locally  optimal  detector 
structure  is  derived  for  this  class  and  the  practical  aspects  of  obtaining 
detector  sub-systems  are  considered.  Examples  of  this  detector  are 
presented  for  noise  sources  with  Laplace  and  Pearson  Type  VII  marginal 


densities,  and  Monte  Carlo  simulations  are  included  to  aid  performance 
enalysis.  A  second  class  of  multivariate  densities  generated  by  a  linear 
transformation  of  an  iid  noise  source  is  also  considered,  and  its  locally 
optimal  detector  is  described. 

In  Chapter  6  the  likelihood  ratio  and  locally  optimal  detectors  for 
several  classes  of  multivariate  densities  are  given.  These  classes  include 
closed  form,  differential  equation,  spherically  symmetric,  series  expan¬ 
sion,  transformation  and  moving  average  models.  Applicability  of  the 
noise  models  and  tractability  of  the  resulting  detectors  are  discussed. 

The  Matched  Filter  (MF)  is  well  known  to  be  the  linear  detector  that 
has  the  maximum  output  Signal-to-Noise  Ratio  (SNR).  In  Chapter  7,  the 
problem  of  finding  the  minimum  filter  length  in  discrete  time  to  achieve 
a  certain  level  of  performance  is  considered  when  there  is  some  freedom 
in  choosing  a  signal  shape.  Upper  and  lower  bounds  on  the  SNR  are  given 
in  terms  of  the  eigenvalues  of  the  noise  covariance  matrix.  Since  these 
bounds  are  rather  difficult  to  compute,  looser,  but  easier  to  compute, 
bounds  are  given.  Several  examples  are  presented  which  illustrate  the 
exact  and  approximate  bounds. 

In  Chapter  8,  the  main  results  of  this  work  are  summerized,  and 
ideas  for  further  research  are  given. 
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CHAPTER  2  -  ASYMPTOTIC  P 


RMANCE  OF  DETECTORS 


WITH  NON-ZERO  INPUT  SNR. 

INTRODUCTION 

Frequently,  the  performance  of  locally  optimal  and  suboptimal 
detection  schemes  is  evaluated  using  asymptotic  measures  such  as  the 
Asymptotic  Relative  Efficiency  (ARE).  Unfortunately,  the  ARE  requires 
the  assumption,  not  only  of  a  large  number  of  samples  (for  the  central 
limit  theorem  to  hold),  but  of  a  vanishingly  small  input  signal-to-noise 
ratio  (SNR).  It  follows  that  any  optimal  or  suboptimal  system  designed  by 
maximizing  ARE  may  perform  poorly  for  a  signal  level  greater  than  zero. 

From  a  survey  of  the  literature,  it  is  evident  that  these  assumptions 
have  concerned  other  researchers  in  the  past.  In  particular,  Miller  and 
Thomas  [1]  investigated  the  convergence  of  Relative  Efficiency  to  ARE  for 
a  signal  in  iid  Laplace  noise.  They  found  the  convergence  to  be  relatively 
slow,  implying  that  the  ARE  may  not  be  an  accurate  measure  for  small  to 
moderate  sample  size.  Spaulding  [2]  found  that,  for  small  sample  size 
and  large  signal  level,  the  hard  limiter  may  well  outperform  the  locally 
optimal  detector  for  a  signal  in  Hall-distributed  noise. 

These  results  motivate  a  further  investigation.  In  this  chapter,  the 
asymptotic  performance  of  several  simple  detector  structures  is  con¬ 
sidered  when  input  SNR  is  not  constrained  to  be  zero.  Originally,  we 
looked  at  the  single  case  of  Laplace  noise  and  were  interested  in  the  rela¬ 
tive  performance  of  the  locally  optimal  (sign),  the  Neyman-Pearson 
(amplifier  limiter),  and  linear  detectors  as  the  input  SNR  is  increased. 
Since  these  are  frequently  encountered  suboptimal,  robust  or 
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nonparametric  detectors,  a  comparison  of  their  performance  for  other 
families  of  densities  might  prove  interesting. 

The  detector  structure  considered  in  this  chapter  consists  of  a  zero 
memory  nonlinearity  (ZNL)  followed  by  a  summer  and  a  threshold  com¬ 
parator  as  shown  in  Fig.  1.  Both  the  likelihood  ratio  and  locally  optimal 
detectors  for  iid  noise  and  constant  signal  have  this  configuration,  and  for 
this  reason  among  others,  this  form  has  particular  appeal  not  only  in 
optimal  but  in  suboptimal  detector  design.  In  this  chapter,  three  detec¬ 
tor  ZNL’s  are  used:  the  linear  amplifier,  the  sign  detector  and  the 
amplifier  limiter. 

DETECTION  PROBLEM 

The  problem  of  detecting  a  constant  signal  in  additive  noise,  given  a 
sequence  of  observations,  can  be  expressed  as  an  hypothesis  and  an 
alternative: 

Ho : 

Hj:  =  rii  + 

Under  the  hypothesis  Ho,  the  observation  Xj  consists  of  noise  only;  under 
the  alternative  Hj  it  consists  of  noise  plus  an  additive  signal  if.  When  the 
noise  is  assumed  to  be  independent  and  identically  distributed  (iid)  with 
density  /  (n*),  both  the  likelihood  ratio  detector  and  the  locally  optimal 
detector  consist  of  a  ZNL  g  (•)  followed  by  a  linear  filter.  If  the  number  of 
samples  is  m  then  the  test  statistic  Tm  is  given  by 

rw  =  Ep(**) 

<-i 

To  simplify  notation  somewhat,  let 
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Mo  ~  EqT„ 


=  E0Tm*  -  E<?T„ 


m  =  EtTm  of  =  Etrm2  -  «,*rm 

where  Eq  and  E+  represent  the  expected  value  under  Ho  and  Hj  respec¬ 
tively. 

If  m  is  sufficiently  large  for  the  central  limit  theorem  to  be  applied, 
then  under  Hq  and  Hj  the  two  random  variables 

vm  Oq  vm  04 

are  both  approximately  unit  normal,  and  the  approximate  level  a  is 
therefore 

a-  f  d$(x)  =  ♦  — - 

t4nto  Vmo0 

Vm  aj) 

•where  $  is  the  Gaussian  N(0,1)  cdf.  The  threshold  t  can  be  found  in  terms 
of  a  : 

t  =  m^o  ”  Oo$_1(«) 

Likewise  the  approximate  power  can  be  computed: 

fi  -  ?  d*(x)  =  *  — * 

t  -  m/if  >/rn  O4 

Vm«f 

Using  the  expression  for  the  threshold  yields  the  power  function: 

0(a)  =  *[££4-»(a)  + 

[a,* 

The  approximate  power  of  a  test  for  fixed  level  a  and  large  sample  size  m 
is  a  monotone  increasing  function  of  the  square  root  of  the  detector  out¬ 
put  SNR,  where 
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_  (MS  ”  Mo)2 
ONKOUT - 3 - 

As  Vm  -*  ■«»,  the  term  containing  the  square  root  of  the  SNR  will  clearly 
dominate  the  other  term  in  the  power  function,  and  the  SNR  is  an  accu¬ 
rate  measure  of  detector  performance.  For  moderate  m,  where  the 
Gaussian  assumption  may  still  be  quite  good,  the  term  Oq$~1(<x)/  a*  can 
become  important.  In  this  case,  using  the  SNR  to  compare  the  perfor¬ 
mance  of  two  tests  may  not  be  enough;  Oq/  should  ideally  equal  the 
same  constant  for  both  tests.  In  the  following  examples  o0  =  o+  for  all 
detectors  considered.  Asymptotically,  the  results  are  equivalent,  but  this 
should  yield  more  valid  results  for  moderate  m . 

COMPARISON  OF  SEVERAL  TESTS 

In  this  section,  three  tests  are  considered:  the  linear  detector,  sign 
detector  and  the  amplifier  limiter.  Under  H0  the  observation  x  has  distri¬ 
bution  F(x)  and  under  Hi  distribution  Obviously,  the  noise  vari¬ 

ance  will  have  no  effect  on  the  gain  in  SNR  of  each  detector  (input  to  out- 

« 

put),  and  therefore  it  can  be  set  equal  to  unity  with  no  loss  of  generality. 

1.  The  linear  detector  (LD)  is  simple  and  frequently  used.  It  is  usually 
justified  by  making  a  Gaussian  assumption  about  the  noise,  and  is  both 
the  Neyman  Pearson  and  locally  optimal  detector  for  Gaussian  noise.  The 
LD  has  the  ZNL 

F(*)  =* 

and  so  ' 

Mo  =  0  Me  =  *0  and  cr02  =  <7^=1 
and  the  output  SNR  of  this  detector  is 


SNRW  =  tJ2  (1) 

2.  The  sign  detector  (SD)  is  probably  the  simplest  ZNL  which  may  be 

effective  in  an  impulsive  environment.  It  has  been  shown  [4]  that  the  SD 
frequently  performs  reasonably  well  for  small  signals  in  impulsive  noise. 


The  SD  has  the  ZNL: 


p(z)  =  sgn(x ) 


Thus 


Mo  =  1  -  2/’(0)  Mo  =  1  -  ZF( — t?)  and  oq  =  o*=  1 

and  the  SNR  of  the  SD  is 

SNRsd  =  4(F(0)  -  F(-ti)]z  (2) 

3.  The  form  of  the  amplifier  limiter  (AL)  used  here  is  the  Neyman- 

Pearson  optimal  test  ZNL  for  Laplace  noise,  with  breakpoints  at  the  signal 
level  0.  While  this  is  not  necessarily  the  best  breakpoint  for  all  noise 
environments,  it  avoids  the  difficulties  inherent  in  choosing  optimal 
breakpoints  [4]  while  still  preserving  some  of  the  robust  characteristics 
described  by  Martin  and  Schwartz  [8]. 


The  AL  has  the  ZNL 


0  x  <  0 

g{x)  =  x 

i?  tf-sx 


Thus 


Mo  -  “  W)]  +  /*  ***(*) 

o 

o 

Mo  =  0[1  ~  /’(—'$)]  +  fx  dF(x) 

-4 

o08  =  T*8  dF(x)  +  t?2[  1  -  F(-t?)]-/uo2 
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w  W 

o*  =  fx2  dF(x)  +  2i>  fx  dF{x)  +  tf2[l-F(-"fl)]  -  ^2 

-a  -tf 

Denote  the  ith  incomplete  moment  as  7* ,  "where 


Ii  =  /x*  dF(x) 
0 


Then  if  /  is  assumed  symmetric: 


fio  =  [&  -  70]  +  /j 

A**  =  +  70]  -  / 1 

a02  =  o«2  =  /2  +  tJ2[£  +  70]  ~  2i>7|  -  Me2 


The  AL  has  SNR 


SNR^  = 


_ 4^70-7^ _ 

72  +  i>2/  4  -  t>7,  -  i32702  -  7j2  +  2t?707i 


RESULTS 


A.  GENERALIZED  GAUSSIAN 


The  first  distribution  considered  is  the  generalized  Gaussian. 


For  a  variance  of  unity 

az/c  =  r(3/c)/r(l/c) 

In  addition  to  including  the  Gaussian  and  Laplace  as  special  cases,  the 
generalized  Gaussian  offers  considerable  control  over  the  rate  of  tail 
decay  through  decay  parameter  c .  It  has  been  suggested  as  a  model  of 
impulsive  atmospheric  noise,  particularly  for  0. 1  <  c  <  0.6,  and  used  to 
demonstrate  the  locally  optimal  detector  by  Miller  and  Thomas  [3],  and 
later  by  Lu  and  Eisenstein  [5]. 


The  incomplete  moments  7*  (for  i  =  0,  1,  2)  are 


2[r(i/c)r(3/c)]» 


r  _  £&/mVjUV 

2  2r(3/c) 

where  y(.,.)  is  the  incomplete  y  function. 

Using  the  incomplete  moments  and  Eqs.  (1-3),  the  output  SNR’s  of 
the  LD,  SD  and  AL  detectors  are  computed  and  plotted  in  Figs.  2-6  for 
several  values  of  decay  parameter  c .  Two  values  of  c  are  of  particular 
interest. 

When  c  =  1  (Fig.  3)  the  generalized  Gaussian  reduces  to  the  Laplace 
density.  The  SD  is  then  locally  optimal  and  the  AL  is  Neyman-Pearson 
optimal.  As  expected,  the  AL  outperforms  both  the  LD  and  SD  for  when 
the  input  SNR  is  greater  than  zero.  Perhaps  surprisingly,  the  perfor¬ 
mance  of  the  locally  optimal  SD  decays  significantly  as  input  SNR 
increases.  In  fact,  the  LD  outperforms  the  SD  when  SNRin  >  0.272. 

The  Gaussian  density  results  when  c  =  2  (Fig.  5.).  In  this  case,  the  LD 
is  both  locally  and  Neyman-Pearson  optimal,  and  as  expected,  it  outper¬ 
forms  both  the  AL  and  the  SD. 

The  points  of  greatest  interest  in  Figs.  2-6  are  the  crossover  points  at 
which  two  tests  have  equal  output  SNR.  In  Fig.  7,  the  contours  of 
SNRsd  ~  SNRld  and  SNRj^  =  SNRu)  are  plotted  in  the  c  xSNRw  plane. 
The  two  contours  divide  the  portion  of  the  plane  considered  into  three 
regions. 

When  SNRm  -  0,  the  SD  and  AL  are  equivalent;  for  all  SNRin  >  0,  the 
AL  outperforms  the  SD.  As  configured  here,  the  AL  requires  the  signal 
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level  -6  to  be  known,  and  when  ■&  is  known  the  AL  would  normally  be  used 
over  the  SD.  The  LD  is  to  be  preferred  over  the  AL  only  in  the  rightmost 
region,  right  of  the  contour  SNRj^  =  SNRjjj.  When  iS  is  not  known,  the 
choice  is  between  the  LD  and  the  SD.  The  LD  outperforms  the  SD  in  all 
but  the  region  left  of  the  contour  SNRSd  -  SNRU >. 

In  Fig.  6,  the  the  relative  performance  of  the  SD  and  the  LD  is  shown 
as  contours  of  ( SNR$d /  SNRm ) &  in  the  cxSNRin  plane.  The  QdB  line 
corresponds  to  the  line  of  equal  performance  in  Fig.  7,  and  performance 
of  the  SD  relative  to  that  of  the  LD  can  be  seen  to  fall  off  quickly  not  only 
with  the  transition  from  Laplace  (c  =  1)  to  Gaussian  (c  =  2)  but  with 
increases  in  SNRjn. 

In  practice,  one  might  use  single  or  (especially  in  nonstationary  noise 

environments)  multiple  estimates  of  c  and  SNRin  and  Fig.  7  to  choose  a 

detector  structure.  To  simplify  the  process,  since  c  is  rather  difficult  to 

estimate,  the  two  contours  SNRsjj  =  SNRid  and  SNR -  SNRjj)  are 

given  in  the  pzxSNRItf  plane  in  Fig.  9.  Here  Pz  is  the  normalized  fourth 
« 

moment,  or  kurtosis  of  the  noise,  frequently  used  as  a  measure  of  density 
"peakedness".  The  kurtosis  is  defined  to  be: 

Pz  =  H 4/mI 

where  a*2  and  are  the  second  and  fourth  noise  moments  about  the 
mean.  Not  only  is  the  kurtosis  pz  easier  to  estimate,  but  it  is  a  more  gen¬ 
eral  measure  than  c  and  can  be  applied  to  many  other  densities. 


B.  PEARSON  VH 


The  next  model  considered  is  the  Pearson  VII  density: 


/*(*)  =  K(\  +  ax2)"m 


IW(m-fc)  -  Z^2 

Rather  than  exponentially  decaying  tails  as  in  the  previous  example,  the 

Pearson  VII  density  has  tails  which  decay  algebraically.  In  the  limit  as 

02  -*  3  the  density  approaches  the  Gaussian  density. 

In  this  case  the  incomplete  moments  /*  (for  i  =  0,  1,  2)  are  computed 
numerically.  Using  these  incomplete  moments  and  Eqs.  (1-3),  the  output 
SNR’s  of  the  LD,  SD  and  AL  were  found  for  several  values  of  kurtosis,  02. 
and  plotted  in  Figs.  (10-13). 

At  02  =  3.  the  Gaussian  density  results,  and  the  performance  of  the 
three  detectors  is  described  in  the  last  section,  and  shown  in  Fig.  5.  As  02 
increases,  the  density  becomes  heavier  tailed  and  the  performance  of 
both  the  AL  and  the  SD  improves  relative  to  that  of  the  LD.  As  before,  the 
AL  consistently  outperforms  the  SD. 

In  Fig.  14,  the  contour  of  equal  performance  of  the  SD  and  LD, 
SNRsd  -  SNRu),  is  shown  in  the  02 xSNRw  plane.  Considering  the 
results  of  the  last  section,  it  is  interesting  how  poorly  the  SD  performs  in 
this  particular  noise  environment.  Comparing  Figs.  9  and  14,  not  only  is 
the  SD  for  Pearson  VII  noise  much  more  sensitive  to  increases  in  SNRjff 
but  the  kurtosis  must  be  nearly  three  times  as  great  (at  SNR{N  =  0)  for 
the  SD  to  outperform  the  LD. 
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C.  MIXTURE  MODEL 

The  next  noise  model  considered  is  a  mixture  model.  Under  certain 
conditions  [4,6,7]  a  noise  process  may  be  well-described  by  a  mixture  of 
the  form 


/  (x)  =  (1  -  e)/*(x)  +  e  //(x) 

where  fg  describes  a  background  noise  process  contaminated  by  bursts 
of  impulsive  fj  distributed  noise.  In  the  examples  which  follow,  the  back¬ 
ground  noise  is  assumed  Gaussian: 


fB(z)  =  vk exp 

The  impulsive  contaminant  is  assumed  to  be  Laplace  or  Gaussian. 


2c2 


1.  Gauss-Laplace  mixture 

The  Gauss-Laplace  mixture  assumes  the  contaminating  impulsive 
noise  to  be  Laplace: 


//(*)  =  |-exp(-a|x  |) 

The  ratio  of  the  variances  7=2/  oPo2  proves  to  be  an  important  constant. 

The  incomplete  moments,  /*  can  be  computed  numerically,  and  the 
resulting  SNR's  can  be  found  from  Eqs  (1-3).  The  contours  of  equal  SNR 
for  the  linear  and  sign  detector  are  plotted  for  several  values  of  7  in  Fig. 
15.  The  SD  outperforms  the  LD  below  the  contours,  and  the  LD  is  superior 
above.  Typically  7  will  be  large  and  c  small  corresponding  to  infrequent 
large  energy  bursts.  In  this  case,  the  sign  detector  may  be  of  use  as 
shown  by  the  7  =  100  contour. 
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2.  Gauss-Gauss  Mixture 


In  this  case  the  contaminant  is  assumed  also  to  be  Gaussian,  with 
variance  s2.  The  ratio  of  the  variances  again  proves  useful,  where 
7  =  s2/  cr2. 

The  contours  of  SNR§d  -  SNRld  are  plotted  in  Fig.  16  for  several 
values  of  y.  The  SD  outperforms  the  LD  below  the  contours  as  in  the  last 
example.  It  is  interesting  to  note  that  while  the  LD  is  optimal  at  e  =  0  or 
t  -  1  since  /  is  then  purely  Gaussian,  the  performance  of  the  SD  may  in 
fact  surpass  that  of  the  LD  for  0  <  t  <  1.  As  stated  above,  y  will  fre¬ 
quently  be  large  and  t  small;  in  this  case  the  SD  would  appear  to  perform 
quite  well. 
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CONCLUSIONS 

Care  should  be  taken  in  applying  locally  optimal  results,  since  the 
locally  optimal  detector  is  designed  for  zero  input  SNR.  As  shown  in  the 
Laplace  example,  the  performance  can  deteriorate  rather  quickly  as 
SNRjjf  increases,  so  quickly  in  fact  that  the  linear  detector  may  well  out¬ 
perform  the  locally  optimal  detector  for  SNRjff  fairly  small. 

The  comparison  of  generalized  Gaussian  and  Pearson  VII  results  indi¬ 
cates  the  importance  of  correctly  choosing  a  noise  model.  The  sign 
detector  performs  considerably  less  well,  particularly  in  sensitivity  to 
increases  in  SNRjf/,  for  Pearson  VII  distributed  noise.  From  these 
results,  we  can  conclude  that  even  two  densities  which  have  equal 
moments  up  to  fourth  order  can  yield  decidedly  different  performance 
from  the  same  detectors. 

In  general  the  performance  of  the  amplifier  limiter  is  quite  good  for 
the  noise  models  used.  The  sign  detector  on  the  other  hand  is  at  times 
disappointing.  One  case  in  which  it  does  appear  to  work  well  was  for  the 
mixture  model  with  infrequent  but  high  energy  impulsive  bursts  (7  large, 
e  small).  Not  only  does  the  sign  detector  outperform  the  linear  detector, 
but  it  is  relatively  insensitive  to  the  form  of  the  contaminant;  similar  per¬ 
formance  is  seen  for  both  Laplace  and  Gaussian  contaminant. 
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CHAPTER  3  -  DETECTOR  DESIGN  USING  A  DENSITY  FIT 
TO  NON-GAUSSIAN  NOISE 

INTRODUCTION 

The  locally  optimal  detector  has  been  suggested  for  the  detection  of 
small  known  signals  in  noise.  The  detector  consists  of  a  nonlinearity  fol¬ 
lowed  by  a  linear  filter  and  a  threshold  comparator  (Fig.  1).  In  this 
chapter,  we  consider  the  selection  of  suitable  nonlinearities  when  a  com¬ 
plete  statistical  description  of  the  noise  is  not  available. 

In  practice,  the  realization  of  the  locally  optimal  nonlinearity  may 
present  some  difficulties.  The  functional  form  may  be  unwieldy  to  imple¬ 
ment.  Especially  for  dependent  noise,  a  multivariate  density  may  not  be 
known  with  precision  and  it  may  be  difficult  to  estimate.  The  noise  may 
not  be  stationary,  making  it  desirable  for  the  nonlinearity  to  adapt  to 
changing  noise  statistics. 

For  independent,  identically  distributed  (iid)  noise,  the  nonlinearity 
is  memoryless,  and  suboptimal  approximations  to  this  zero  memory  non¬ 
linearity  (ZNL)  have  received  some  attention.  Kassam  and  Lim  [4]  have 
addressed  the  problem  of  finding  an  optimal  quantized  version  of  the 
ZNL.  Miller  and  Thomas  [2,3]  use  hard  limiters,  amplifier  limiters,  mul¬ 
tilevel,  and  piecewise  linear  suboptimal  nonlinearities  in  place  of  the 
optimal  ZNL.  Frequently  these  approximations  have  parameters  which 
are  chosen  to  maximize  the  detector  efficacy.  Unfortunately,  this  usually 
proves  to  be  a  numerical  problem  involving  considerable  computations. 
It  is  not  always  clear  how  to  choose  or  optimize  these  nonlinearities  when 


the  noise  density  is  not  known  exactly. 


A  second  approach  is  used  in  this  chapter.  An  approximate  non¬ 
linearity  with  a  desirable  functional  form  and  one  or  more  free  parame¬ 
ters  is  chosen.  A  family  of  noise  densities  with  the  same  free  parameters 
can  then  be  found  for  which  the  nonlinearity  is  optimal.  The  parameters 
are  evaluated  by  fitting  a  member  of  this  family  of  densities  to  the  noise. 


LOCALLY  OPTIMAL  DETECTOR 

The  problem  of  detecting  a  known  signal  s  in  noise  n  can  be 
expressed  as  an  hypothesis  Hq  and  an  alternative  Hx\ 


Hq.x  -  n 

H\.t  =  n  +  •ds  >  0 

Under  H0,  the  observation  vector  x  consists  of  noise  n  with  density  /  (n), 
and  under  H  j  the  observation  consists  of  noise  plus  known  signal  s  with 
amplitude  tJ.  The  detector  can  be  written  as  a  real-valued  functional  T(x) 
on  the  observation,  followed  by  a  threshold  comparator  to  decide  for  Hq 
or  H\. 

When  the  magnitude  tJ  of  the  signal  is  unknown,  but  assumed  to  be 
nearly  zero,  the  efficacy 
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is  frequently  used  as  a  measure  of  detector  performance  [1-8].  It  is  well 
known  [14]  that  the  statistic  Tu  which  maximizes  the  efficacy  is  the  tJ 
derivation  of  the  log  likelihood  function  at  tf=0: 
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known  as  the  locally  optimal  detector.  For  a  signal  in  additive  noise  with 
amplitude  the  statistic  consists  of  a  nonlinearity 

9u>(x)  =  ~V/(x)//(x) 

followed  by  a  correlator 

^(x)  =  -  ^  ^'S  ~  ffta(x)  s 

where  "V"  is  the  gradient  and  is  the  vector  dot  product.  The  statistic 
reduces  for  iid  noise  to  a  zero  memory  nonlinearity  (ZNL) 

ffto(x)  = -/'(x)//(x)  (2) 

followed  by  a  correlator: 

7io(*)=  E-si/'(*i)//(*i)  =  Esi0to(zi) 

**1  t*l 

with  efficacy 

Ar„)  =  /(/  *//)& 

Suppose  a  ZNL  g  is  chosen  with  free  parameters  i>.  Then  Eq.(2)  can 
be  viewed  as  a  differential  equation  which  can  be  solved  under  certain 
regularity  conditions.  The  differential  equation  is  written  as 

=  -/'(*)//(*) 

Thus 

fg(x)dx  =  -fd\n(f(x))  -  -ln(f  (x))  +  C 

and  so 


f(x)-K  exp' 


-fg{x)dx 


A  suitable  suboptimal  ZNL  can  now  be  selected,  and  the  correspond¬ 
ing  family  of  solution  densities  found  for  which  the  ZNL  is  optimal.  Not 
only  should  the  ZNL  have  a  relatively  simple  form,  but  it  must  lead  to  a 


reasonable  set  of  densities.  Since  a  member  of  the  solution  family  is  to 
be  fitted  to  the  observed  noise,  it  should  be  general  enough  to  provide  a 
relatively  good  fit,  and  the  densities  should  have  parameters  that  are  not 
too  difficult  to  estimate. 

PEARSON  APPROXIMATION  TO  THE  ZNL 

As  an  approximate  ZNL  consider  a  rational  function: 

g(x)  = -N(x)/ D(x)  (3) 

where  N  and  D  are  polynomials  in  x  of  degree  n  and  d  respectively.  This 

is  a  general  form  of  the  Pearson  differential  equation. 

Choosing  n  =  1  and  d= 2  yields  the  classical  Pearson  family  as  solu¬ 
tions  of  the  differential  equation: 

Lixl  .= - -  (4} 

/(*)  60+{>i*+b2*2 

The  classical  Pearson  family  includes  as  special  cases  the  Gaussian. 
Cauchy,  t,  F,  x2,  uniform,  gamma,  exponential  and  beta  densities.  Origi¬ 
nally  proposed  by  Karl  Pearson  [11]  as  a  tool  for  fitting  densities  to  data, 
it  has  continued  to  attract  considerable  attention. 

Solutions  of  the  differential  Eq.(4)  depend  on  the  roots  of  the  denomi¬ 
nator  D.  There  are  three  possibilities: 

1.  real  roots,  same  sign 

2.  real  roots,  different  sign 

3.  complex  conjugate  roots 

These  correspond  to  the  three  main  Pearson  types  VI,  I  and  IV  respec¬ 
tively.  Transition  types  result  when  parameters  in  the  main  types 
approach  a  particular  set  of  limiting  values.  The  Pearson  types  and 
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associated  common  densities  are  summarized  in  Table  1. 

Maximum  likelihood  (ML)  estimators  of  the  parameters  in  the  Pear¬ 
son  family  depend  on  the  density  type  which  is  the  best  fitted  to  the 
observations.  A  simpler  approach  is  to  use  the  method  of  moments  (MM) 
which  requires  estimates  of  the  first  four  noise  moments.  For  nearly 
Gaussian  noise  (0,  ^0.1  and  2.62  ^  02  ^  3.42),  the  ratio  of  the  variance  of 
MM  to  ML  estimates  exceeds  80%  [12,13]. 

The  coefficients  of  the  Pearson  system  can  be  found  in  terms  of  the 
first  four  moments  (or  skewness  and  kurtosis)if  they  exist  [9, 10, Appen¬ 
dix].  Define  variance,  skewness  and  kurtosis: 

o2  =  M2  .  Pi-  M32/M23  .  Pz  =  M4/  Vv 

In  terms  of  these  quantities  the  coefficients  are: 

—a  —  6j  =  £*,*082+3  )/D 
b0-  -02(4/52-3/5,)/  D 
*2  =  (2/52-3/5,— 6)/ £ 
with  the  denominator  D  given  by 

D  -  10/52“  12/3,— 18 

As  an  aid  to  selecting  a  Pearson  type  density  given  the  sample  values 
of  0,  and  02,  Pearson  constructed  a  chart  showing  the  regions  occupied 
by  each  of  the  Pearson  types  in  the  0,x02  plane  (Fig.  2). 

Although  the  Pearson  family  consists  of  both  skewed  and  symmetric 
densities,  only  the  symmetric  (0,=O)  are  considered  below.  The  sym¬ 
metric  solutions  can  be  seen  from  Fig.  2  to  include  the  Pearson  types  II, 
VII  and  normal  densities.  These  can  be  written  in  a  single  function  form: 

fix)  =  ATl-ax8)*1 
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The  constant  K  can  be  written  in  terms  of  the  incomplete  beta  function 

*(-): 


When  the  first  four  moments  exist: 


K  - 


m  >  —  1  II 

m  <  1  VII 


a  =  (3-02)/  2ct202  (5) 

m  -  (502-9)/2(3-02) 

Note  that  for  02<3,  /  (x)  has  finite  support,  |x  |  <a~%.  For  02^3,  /  (z)  has 
infinite  support,  and  at  02=3,  the  density  is  normal.  Fig.  3  shows  /(x) 
for  several  values  of  02,  and  properties  of  /(x)  are  summarized  in  Table 
2. 


Restricting  the  density  to  be  symmetric  simplifies  the  ZNL  since, 
from  Eq.(ll),  when  01=0,  then  a=b  5=0  and  the  resulting  ZNL  is  odd  sym¬ 
metric.  The  locally  optimal  ZNL  is  found  from  Eq.(2)  to  be: 


*!.(*)= 

1-ax* 


(6) 


with  range  — °°<x<°°  for  a^O  and  jxj for  a>0.  The  parameter  a  in 
this  nonlinearity  can  be  estimated  from  the  variance  and  kurtosis  of  the 
observed  noise  using  the  expression  in  Eq.(5).  The  II/VII  ZNL,  shown  in 
Fig.  4  for  several  values  of  02,  can  take  three  important  forms:  expander 
(02<3),  linear  (02=3).  and  contractor  (02>3).  A  fourth  type,  the  limiter,  is 
not  included,  causing  some  problems  when  fitting  to  a  density  with 
exponential  tails,  for  which  a  limiter  is  optimal. 

The  Asymptotic  Relative  Efficiency  (ARE)  is  defined,  under  certain 
regularity  conditions,  to  be  the  ratio  of  the  efficacies  of  two  detectors: 


ABE1i2  =  J(T1)/J(T2) 


and  is  a  frequently  used  measure  of  relative  performance  of  the  detec¬ 
tors.  A  linear  detector  for  any  density  is  easily  seen  from  Eq.(l)  to  have 
the  efficacy- 

J(Tuj)  =  l//^2 

The  efficacy  of  the  locally  optimal  detector  of  the  Pearson  II /VB  density 
is 

J(T  x  5(fa-3)(2fa-3) 

Wto;"  M2/32(7,52-15) 

The  AREio  u,  is  plotted  in  Fig.  5.  The  ARE^^  is  a  minimum  of  unity  at 
02  =  3  corresponding  to  the  Gaussian  density.  As  02  decreases,  resulting 
in  densities  with  increasingly  lighter  tails,  the  ARE  increases  asymptoti¬ 
cally  to  infinity  at  02  =  15/7.  For  heavier  tailed  densities,  the  improve¬ 
ment  is  less  marked.  As  02  -*  00  the  ARE  approaches  the  limiting  value 


Fig.  3.  Pearson  Il/VII  density 
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EXAMPLES 

In  the  last  section,  a  method  of  finding  a  suboptimal  ZNL  using  the 
first  four  sample  moments  was  developed.  In  practice  we  would  use  only 
these  moments  to  fit  our  suboptimal  ZNL.  The  problem  of  estimating 
these  moments  efficiently  for  an  unknown  density  will  not  be  considered 
here.  Instead,  in  our  examples,  we  assume  two  classes  of  non-Pearson 
noise  densities  and  use  analytically  derived  moments  to  fit  the  Pearson 
ZNL.  Then  the  efficacies  of  the  suboptimal  and  optimal  detectors  are 
compared  as  a  measure  of  detector  performance.  The  two  classes  of  den¬ 
sities  chosen  are  the  generalized  Gaussian  and  a  Gauss-Laplace  mixture 
density. 

A.  Gauss-Laplace  Mixture  Noise 

The  Gauss-Laplace  density  is  given  by: 

/(l)  =  &Srexp[iS] +  f*xp(-,,|l|) 

where  Os;eil  and  0 Ka2,^.  One  justification  of  this  density  is  to  assume  a 
Gaussian  noise  with  bursts  of  Laplace  (impulsive)  noise  100e%  of  the  time 
[2].  An  additional  quantity  which  proves  useful  is  the  ratio  of  the  vari¬ 
ances: 

7  =  (i?o)z/  2 

Typically  e  and  y  are  assumed  small.  The  moments  are: 

li\  =  H 3  =  0 

M2  =  (l-e)o2  +  e(2/'i?2) 

1*4  =  (I-c)it2  +  e(  2/tf2) 
with  skewness  and  kurtosis: 
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Pz  ~ 
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(1-eVy2  +  2c 
[(l-e)7  +  e]2 


The  Pearson  ZNL  is  given  by  Eq.(6): 


^(z)  =  x/fl-ax2) 
where  a  can  be  found  from  Eq.(5): 


o  -  (fig— 3)/  iZvofizl 

Notice  that  the  Pearson  ZNL  is  designed  using  estimates  of  the  noise 
moments,  and  not  estimates  of  the  quantities  e  or  y  which  are  more 
difficult  to  compute.  Since  the  moments  are  relatively  simple  to  esimate, 
this  nonlinearity  is  by  nature  adaptive. 

AREbM  is  computed  numerically  and  shown  with  ARE^  ,u  in  Figs.  6- 
10.  for  several  values  of  7.  Reasonable  performance  is  seen  for  e  small 
(nearly  Gaussian  noise)  even  though  the  Gaussian  noise  is  contaminated 
with  a  non-Pearson  density.  In  addition  the  performance  in  all  cases 
equals  or  exceeds  that  of  the  linear  detector,  AREg  ld>  1. 


B.  Generalized  Gaussian  Noise 


The  Gaussian  density  can  be  generalized  by  providing  for  a  variable 
rate  of  exponential  decay  [l]: 


f(x)  =  AexpHCx  |c) 

where 


<r  = 


ro/ci 

o^d/c) 


and 


K  =  c<-/2r(l/c) 


This  class  of  densities  is  of  interest  because  it  contains  both  the  Gaussian 
(c=2)  and  Laplace  (c=l)  densities  as  special  cases,  and  it  allows  consid¬ 
erable  control  over  the  rate  of  tail  decay. 


The  moments  are  zero  for  n  odd  and  for  n  even: 


=  ^mn+ivcmi/ci2 
r(3/n)n/c 

with  skewness  and  kurtosis: 

p,=0  and 

The  Pearson  ZNL  is  given  in  Eq  (6)  and  a  in  (5).  Once  again,  the  moments 
are  estimated,  and  not  the  decay  parameter  c ,  which  is  rather  difficult  to 
estimate.  Since  a  is  easily  estimated,  the  Pearson  ZNL  is  potentially 
adaptive. 

AREgM  is  computed  numerically  and  shown  in  Fig.  11.  ARE, and 
AREst  u  are  also  shown  for  comparison.  AREsdld  is  the  ARE  of  the  sign 
detector  (hard  limiter)  vs  the  linear  detector.  The  sign  detector  is 
optimal  for  Laplace  noise  (c  =1)  and  nearly  optimal  for  c*l.  However,  for 
nearly  Gaussian  noise  c  «2,  the  Pearson  detector  is  better.  This  suggests 
that  a  system  including  both  the  sign  detector  and  the  Pearson  detector 
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might  well  surpass  the  performance  of  either  one  alone. 


MORE  COMPLEX  ZNL’s 


Values  of  n  and  d  can  be  chosen  other  than  n  =  l  and  d= 2  of  the  clas¬ 
sical  family.  The  resulting  families  may  contain  multimodal  curves  (for 
instance  n=  2,  d- 3  results  in  a  family  including  bimodal  curves).  Higher 
values  of  /Sj  and  /S2  may  also  be  admitted.  The  major  disadvantage  of 
increasing  n  and  d  is  that  high  order  moments  are  required.  Since  the 
number  of  parameters  to  be  found  is  n+d  +  l,  using  the  method  of 
moments  requires  n+d  +  l  moments.  As  the  moments  become  increas¬ 
ingly  difficult  to  estimate  with  increasing  order,  increasing  n  and  d 
significantly  may  not  be  possible  or  desirable. 

A  single  or  multiple  discontinuities  can  be  allowed  in  the  nonlinear¬ 
ity.  The  result  is  a  family  of  curves  which  consist  of  segments  of  Pearson 
curves  between  the  points  of  discontinuity.  For  a  single  discontinuity  at 


x=0: 


f'(x )  _  (x+a)(l-c  +du(x) 
/(*)  bQ+b^+b&c2 


where 


0  x<0 

“I1*  =  [t  xao 

Solving  this  differential  equation  yields  a  family  of  Pearson  curves  con¬ 
sisting  of  two  half  curves  for  xfeO  and  x<0.  Each  of  the  two  halves  are  of 
the  same  Pearson  type  but  with  different  parameters.  Among  the  possi¬ 
ble  solutions,  two  of  particular  interest  are  the  Laplace  or  double 
exponential  and  the  double  Gaussian  densities. 


As  an  example  where  this  method  of  curve  fitting  fails,  consider  the 


nonlinearity  on  the  open  interval  (xi,x2): 

g(x)  =  as07i(x)|x  |c 
Then  for  c  *1  and  xj<x<x2: 


/(x)=/rexpJ-|^Tj|I|-j 

This  density  is  commonly  called  the  generalized  Gaussian  with  Laplace 
(c=0)  and  Gaussian  (c  =  l)  as  special  cases.  Unfortunately  there  is  no 
good  method  for  estimating  the  exponential  parameter  c ,  and  thus  the 
ZNL  cannot  be  fitted  in  the  manner  used  above. 


Several  multivariate  extensions  to  the  Pearson  family  have  been  con¬ 
sidered  [10,15].  Following  the  same  line  of  reasoning  which  led  to  the 
univariate  Pearson  family,  K.  Pearson  wrote  the  difference  equations 
describing  the  bivariate  hypergeometric  density  as  a  pair  of  differential 
equations: 


and 


dfix.y)  -  cubic  in  x,y  .  ,  x 
dx  quartic  in  x,y 


(?) 


df(x,y)  _  another  cubic  in  *JLf(x  y)  (0) 

dy  quartic  in  x,y  x 

Other  nonlinearities  are  possible,  but  all  suffer  from  the  same  drawbacks. 
It  is  extremely  tedious  to  solve  the  simultaneous  equations  if  the  method 
of  moments  is  used,  and  the  number  of  coefficients  needed  for  a  com¬ 
plete  fit  increases  extremely  rapidly.  In  the  bivariate  case  alone,  there 
are  15  moments  up  to  fourth  order  [15]: 


1.  1 -total  mass. 

2.  2-position  of  mean. 


3  2-variate  standard  deviations. 

4.  1-coeffieient  of  correlation. 

5  ^ -marginal  /S's  (0,,  /S2,  /S2). 

6.  2-third  order  product  moment  coefficients. 

7.  3-fourth  order  product  moment  coefficients. 

Thus  a  complete  fit  up  to  fourth  order  moments  requires  15  coefficients 
in  the  nonlinearities.  Since  there  are  fewer  than  15  in  Eqs.(7-8),  and  in 
fact  in  most  reasonable  nonlinearities,  some  relationship  among  the 
higher  moments  must  be  tolerated  to  reduce  the  number  of  coefficients 
required.  It  then  becomes  less  clear  that  the  fit  will  be  a  good  one. 

CONCLUSION 

The  optimal  ZNL  in  a  locally  optimal  detector  may  be  cumbersome  to 
implement  or  even  impossible  to  find  when  noise  statistics  are  not  known 
exactly.  It  is  possible  to  choose  a  practical  form  for  the  nonlinearity,  and 
solve  the  locally  optimal  differential  equation  yielding  a  family  of  solution 
densities  for  which  the  nonlinearity  is  optimal.  One  of  these  densities  can 
then  be  fitted  to  the  noise,  and  the  corresponding  detector  used.  If  the 
family  of  densities  can  provide  a  good  fit  to  the  noise,  it  is  likely  that  the 
detector  so  derived  will  have  nearly  optimal  performance. 

Choosing  a  rational  function  for  the  ZNL  in  a  locally  optimal  detector 
results  in  a  particularly  useful  family  of  solution  densities,  the  Pearson 
family.  Not  only  does  the  Pearson  family  contain  many  common  densi¬ 
ties,  including  the  Gaussian,  but  for  nearly  Gaussian  noise  the  method  of 
moments  can  be  used  efficiently  to  compute  the  coefficients  of  the  ZNL. 
This  method  produces  a  detector  which  performs  at  least  as  well  as  the 


1 
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linear  detector  for  the  two  classes  of  densities  considered,  and  nearly  as 
well  as  the  optimal  detectors  for  nearly  Gaussian  noise. 


> 
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APPENDIX  -  Method  of  Moments  for  Pearson  Family 

The  coefficients  of  the  Pearson  system  can  be  found  in  terms  of  the 
first  four  moments  if  they  exist  [9,10].  Cross  multiply  the  Pearson 
differential  equation,  Eq.(12),  multiply  both  sides  by  xn  and  integrate  to 
obtain 


fxn(b0+b1x+b^x2)/‘(x)dx  =  Jxn(a+x)/  (x)dx 
The  left  side  can  be  integrated  by  parts: 

xn(b0+b1x+b^xz)f f'(x)dx  - 
f  (n60xn-1+(n  + 1)6  jXn  +(n+2)b^zn*’l)f  (x)dx 
=  afxnf(x)dx+  J‘xn+1f(x)dx 

but  f  f'(x)dx  =  0,  and  Mi*  is  the  i—th  moment  about  zero.  This  gives 
four  equations  (for  n=0,  1,  2,  3): 

-*Mn+l'  =  ttMn'+b0nMn-l'+6l(n  +  1)A‘n'+i’2(rl+2)Mn+r 
Shift  the  moments  to  moments  about  the  mean  by  setting  Mi  =  0  and 

Mi'  =  Mt  for  i  =  2,  3,  4.  Thus 

6j  +  a  =  0 
60  +  3M2t2  ~  ~M2 
3M2&  i  +  4M3b2  +  M2q  =  “M3 
3&>b0  +  4m36i  +  5M4b2  +  M3a  =  "M4 

Solving  gives: 

-a  =  6,  =  Mg(M4+3M22)/£ 

6o  =  M2(3M32-4M2M4)/  D 
6g  =  (-2m8A*4+3M32+0M23)/^ 

£>  =  10mzM4-18M23-12m3Z 
Define  variance,  skewness  and  kurtosis: 


02  =  M2  .  01  =  M32/  M23  .  02  =  /4|/M22 
In  terms  of  these  quantities  the  coefficients  are: 

-a  =  6,  =  £70 ,*(02+ 3)/ £ 

60  =  -ct2(402-30i)/  Z? 

62  =  (202-30,-6)/  Z? 

Z?  =  1002-120,-18 
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CHAPTER  4  -  AN  EXAMPLE  OF  MOMENT  ANALYSIS 
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ON  ARCTIC  UNDER-ICE  DATA 

1N1RODU  CTTON 

As  an  example  of  data  subjected  to  moment  analysis,  we  consider 
samples  of  under-ice  ambient  noise.  The  data  were  collected  on  April  23, 
1930  under  Arctic  pack  ice  at  86°N  latitude  and  25°W  longitude  by  a 
multi-institutional  experimental  group.  The  raw  samples  were  gathered 
using  an  omnidirectional  hydrophone  suspended  91  meters  below  the  ice, 
and  they  represent  approximately  10  minutes  of  noise.  The  noise  was 
prefiltered  with  a  low -pass  filter,  cutoff  frequency  at  2600Hz  and 
96dB/octave  roll-off.  The  result  was  sampled  at  10kHz  and  stored  as  6006 
records  of  1024  samples,  each  record  representing  about  0.1  seconds. 
For  further  information,  the  reader  is  referred  to  References  [1  and  2]. 

COMPUTATION  OF  SAMPLE  MOMENTS 

In  this  chapter,  as  in  [1],  the  Arctic  under-ice  samples  are  assumed 
independent  and  stationary.  The  first  four  sample  moments  are  com¬ 
puted  for  each  record  of  1024  samples  (w  0.1  seconds).  As  an  estimator 
of  the  kth  central  moment,  fj,k ,  consider  the  kth  central  sample  moment, 
mk. 

™*  =  7T  E  (**  “  *)* 

yv  i»l 

where  I  is  the  sample  mean: 

i  N 

x  ~ 

yv  *■! 
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In  the  previous  chapter,  the  skewness  0i  and  kurtosis  02  were  defined 
in  terms  of  the  variance  o2  and  the  third  and  fourth  central  moments 
and 

0i  ~  Us/  o3  and  fi2  =  a 4 

As  estimators  of  a2,  /S,  and  02-  the  sample  variance  s2,  skewness  6j  and 
kurtosis  b2  can  be  used,  where 

s2  =  m2  6 1  =  mf /s3  bz-m^/sA 

The  sample  variance,  skewness  and  kurtosis  are  plotted  versus 
record  number  in  Figs.  1-3  for  the  Arctic  under-ice  ambient  noise  data. 
These  plots  are  essentially  the  same  as  those  in  [  1],  except  that  here  the 
moments  are  centered  on  the  sample  mean  of  each  record,  rather  than 
on  the  mean  of  the  entire  6006  records.  From  these  figures  it  can  be 
seen  that  the  noise  under  study  is  not  only  nonstationary,  but,  at  times, 
non-Gaussian. 

Both  sample  skewness  and  kurtosis  have  been  suggested  [2,4]  as 
measures  of  Gaussianness.  The  1%  and  5%  confidence  intervals  for  6  x  and 
62  given  a  sample  size  1000  are  given  in  Table  1.  as  computed  in  [4]. 
Applying  either  of  these  bounds  to  Fig.  2  or  3  gives  a  clear  indication  that 
the  data  consist  of  Gaussian  or  nearly  Gaussian  noise  with  sporadic, 
highly  non-Gaussian  bursts. 

If  skewness  and  kurtosis  are  taken  to  be  measures  of  Gaussianness,  a 
convenient  tool  for  viewing  these  statistics  is  the  /3jX02  plot.  Originally 
suggested  by  K.  Pearson  [5,6]  as  a  tool  for  curve  fitting,  it  is  discussed  at 
length  in  the  last  chapter.  Its  most  useful  property  is  that  each  density 
or  family  of  densities  corresponds  to  a  specific  region  of  the  /JjXfe  plane. 
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For  example  the  Gaussian  density  corresponds  to  a  single  point. 
(jSi-ftz)  =  (0.3).  Wliile  a  (/Si.fe)  pair  does  not  uniquely  specify  a  density, 
two  densities  at  the  same  point  can  be  made  to  have  equal  moments  up 
to  fourth  order.  The  Pearson  and  Johnson  families  are  of  particular 
interest  since  they  both  cover  the  entire  possible  region  of  the  /SjXftj 
plane. 

The  Pearson  family  is  described  in  the  last  chapter  and  in  [5,6].  It 
contains  the  Cauchy,  t,  F,  x2,  uniform,  gamma,  exponential  and  beta  den¬ 
sities  as  special  cases.  Fig.  4  is  a  /9i*jS2  plot  with  the  regions  correspond¬ 
ing  to  the  Pearson  types  on  it.  Superimposed  on  this  figure  is  a  scatter- 
plot  of  the  (6 1,62)  pairs  generated  for  each  of  the  6006  records. 

The  Johnson  family  is  described  in  [5,6]  and  consists  of  three  types. 
In  the  Johnson  system,  the  transformation  of  a  non-Gaussian  variate  is 
assumed  to  be  unit  normal.  In  the  most  common  of  the  Johnson  types, 
the  lognormal,  SL,  the  transformation  is 

gL(x)  =  c  +  log [ (x  -  a)/ 6  ] 

Thus  if  x  is  lognormal,  gi{x)  is  unit  normal.  In  a  similar  fashion,  Sy  and 
Sg  are  defined  by  the  transformations: 

g-j{x)  =  c  +  d  sinh_1[(i  -  a)/  6  ] 

and 

gB(x)  =  c  +  cilog[(x  -  a)/  (6  -  x)] 

In  Fig.  5  the  sample  (61,63)  pairs  are  plotted  on  the  /5]X/S2  plane  for  the 
Johnson  family. 

In  both  plots,  the  vast  majority  of  the  points  are  clustered  about  the 
point  (0,3)  corresponding  to  the  Gaussian  density  in  both  families.  While 
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confidence  regions  are  extremely  difficult  to  compute,  nevertheless  these 
figures  show  more  clearly  than  Fig.  2  and  3  the  essential  Gaussian  nature 
of  the  data. 


CONCLUSIONS 


The  results  in  Figs.  1-5  indicate  that  the  Arctic  under-ice  noise  is 
non-stationary  and  although  Gaussian  or  nearly  Gaussian,  it  is  contam¬ 
inated  by  sporadic  bursts  of  a  highly  non-Gaussian  nature.  Further 
moment  analysis  of  the  non-Gaussian  component  is  of  questionable  value, 
due  to  the  small  number  of  highly  non-Gaussian  records.  Perhaps  with  a 
larger  data  set,  stronger  conclusions  could  be  drawn. 

Two  assumptions  are  made  earlier  in  this  chapter  should  be  exam¬ 
ined  further.  The  data  are  assumed  stationary,  but  the  results  obtained 
do  not  bear  this  out.  Some  investigation  of  the  degree  of  nonstationarity, 
perhaps  the  rate  at  which  the  moments  change  would  be  of  interest.  The 
data  are  clearly  not  independent,  since,  among  other  sources  of  depen¬ 
dence,  they  were  prefiltered  before  being  recorded.  However,  the 
assumption  of  independence  is  made,  and  the  sample  moment  estimators 
are  perhaps  not  the  best  that  could  be  used.  For  example,  consider  the 
sample  variance: 


•e=4r2(^-*)a=7r£^- 


N 


N 


*  «- 1 

Taking  the  expected  value  yields 


<  N  N 

77 T  £  £  x*z> 

JV  i  =  1  J=  1 


where  the  afj  are  the  covariance  terms.  When  the  noise  is  uncorrelated, 


APPENDIX  -  Moments  Estimators  for  Dependent  Noise. 

The  following  is  an  example  of  moment  estimators  which  have  been 
corrected  for  second  order  dependence.  The  prewhitening  technique 
used  requires  that  the  correlation  either  be  known  or  estimated.  Note 
that  this  corrects  only  to  second  order,  and  the  expected  value  of  higher 
order  moment  estimates  still  may  contain  non-zero  cross-moment 
coefficients  of  order  greater  than  two. 

A  data  record  can  be  written  as  a  length  N  vector  x.  with  yVxTV 
covariance  matrix  £.  Computing  the  moments  about  zero  requires  that 
the  vector  x  be  centered;  this  can  be  written  [4]  in  terms  of  the  centering 
matrix  C: 


where  I  is  the  identity  matrix  and  1  is  a  vector  of  1  ’s. 

The  centered  vector  may  still  be  correlated,  but  it  can  be  prewhi¬ 
tened  if  it  is  assumed  that  the  correlation  matrix  R  =  £/  o2  is  known. 
Assuming  R  is  Toeplitz,  it  can  be  factored  into  an  upper  and  lower  tri- 
angular  matrix: 

R  =  LU  and  R"J  =  U^L"1 

where  L  =  \JT  and  Lf 1  =  (U-1)7'.  The  matrix  L-1  can  be  used  to  prewhiten 
the  centered  matrix  with  the  result  that  L-1Cx  is  uncorrelated.  An  esti¬ 
mate  of  the  ith  moment  about  the  mean  is  given  by: 

m*  =  <*)* 

W  «*1 

where  y  =  L_1Cx  is  the  centered  and  prewhitened  observation  vector.  The 
three  parameters  of  interest  in  the  last  chapter  are  the  variance, 


skewness  and  kurtosis.  The  variance  o2  has  estimate  s 

:S2  = 

N 

The  estimates  of  skewness  and  kurtosis  are 
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CHAPTER  5  -  LOCALLY  OPTIMAL  DETECTION  IN  MULTIVARIATE 
NON-GAUSSIAN  NOISEf 

INTRODUCTION 

In  problems  of  detection  and  estimation  in  discrete  time,  the 
observed  noise  sequence  is  often  assumed  to  be  independent.  Under  this 
assumption,  various  detection  schemes  have  been  found  and  analyzed 
[1,2, 3, 4].  Often,  however,  a  strong  dependence  structure  exists  suggest¬ 
ing  that  designs  based  on  the  assumption  of  independence  are  less  than 
optimum.  Previous  research  on  detection  in  dependent  noise  is  often 
limited  to  simple  detector  structures  such  as  a  linear  (matched)  filter  or 
a  ZNL  followed  by  a  linear  filter  [5].  In  this  chapter,  a  noise  observation 
consists  of  a  length  m  vector  which  is  assumed  to  be  no  more  than  in¬ 
dependent.  Under  this  model  the  noise  statistics  are  contained  in  an  Tri¬ 
dimensional  multivariate  distribution  and  optimal  detectors  often  con¬ 
tain  nonlinearities  with  memory.  Finding  a  suitable  multivariate  noise 
distribution  for  dependent,  non-Gaussian  noise  is  a  problem  with  no  sin¬ 
gle  best  solution.  Instead  we  discuss  several  known  forms  of  multivariate 
densities  and  draw  attention  to  a  characterization  entitled  transforma¬ 
tion  noise  which  has  several  useful  properties. 

There  exist  a  number  of  well  known,  closed  form,  multivariate  densi¬ 
ties  [6,7]  including  the  Gaussian,  Wishart,  multivariate  Pearson  family 
and  multivariate  forms  of  many  common  univariate  densities.  The  mul¬ 
tivariate  Gaussian  is  often  employed  not  only  because  of  its  tractability 

fThis  chapter  (excluding  Appendix  B)  was  co-authored  with  Peter  F.  Swaszek,  and 
also  appears  in  his  PhD  dissertation  Robust  Quantization,  Vector  Quantization 
and  Detection,  Princeton  University,  October  1982. 
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but  because  a  preponderance  of  evidence  suggests  that  it  is  the  natural 
multivariate  density  with  Gaussian  marginals.  Unfortunately  central 
limit  arguments  do  not  always  hold  and  this  approximation  can  be  poor. 
Additional  closed  form  multivariate  densities  can  be  generated  by  replac¬ 
ing  the  argument  of  a  univariate  density  with  the  square  root  of  a  qua¬ 
dratic  form  [8].  This  extends  each  univariate  to  an  elliptically  symmetric 
density  in  m -space.  These  densities  all  have  closed  functional  forms  and 
the  advantage  of  tractability,  but  the  dependence  structure  may  not  be 
well  represented  by  any  of  these  densities. 

A  second  approach  is  to  use  a  multivariate  series  expansion  of  the 
Gram-Charlier  type  [9],  This  has  a  certain  theoretical  elegance  and  can 
be  used  to  derive  general  properties  of  densities.  For  independent  data, 
useful  detection  procedures  have  resulted  from  this  method  since  the 
nonlinearity  is  a  function  of  the  moments  of  the  noise  process  [10].  There 
is  still  the  problem  of  choosing  a  proper  weighting  density  on  which  to 
base  the  series.  By  careful  selection,  it  is  often  possible  to  minimize 
truncation  and  tail  errors. 

In  a  multivariate  setting  however,  it  is  less  clear  how  well  series  will 
work.  For  one  thing,  the  number  of  coefficients  in  a  series  increases 
exponentially  in  m  and  the  expansion  requires  estimates  of  the  cross 
moments  of  the  process.  It  is  necessary  to  truncate  the  series  which  may 
result  in  a  poor  representation  or  in  negative  values  in  the  tail  regions  of 
the  pdf.  The  number  of  high  order,  cross  moment  estimates  requir'  d  for 
good  representation  may  be  prohibitive. 

A  third  class  of  densities  which  are  often  considered  are  those  gen- 
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erated  by  transformations  from  other  densities  [11],  Common  univariate 
examples  are  the  log-normal  and  the  Johnson  family.  In  a  multivariate 
setting,  this  method  involves  a  nonlinearity  (possibly  with  memory) 
operating  on  a  noise  sequence  with  a  known  density.  The  intractability  of 
this  general  class  of  transformations  suggests  constraining  interest  to 
those  invertible  transformations  with  zero  memory  (ZNL).  A  block 
diagram  of  the  system  generating  this  type  of  noise  is  given  in  Fig.  1. 
Given  an  identically  distributed,  m -dependent  background  noise  process 
v  with  a  known  density  <p{v)  and  marginals  y>\(vx),  a  ZNL  g  can  be  selected 
to  produce  a  noise  n  with  the  desired  marginals  /^n*).  If  tp1  and  g  are 
fixed,  the  dependence  structure  of  n  is  completely  determined  by  the 
background  noise  dependency.  This  lack  of  fi  "Ability  in  choosing  a 
dependence  structure  may  be  a  disadvantage  in  some  cases.  Noise  gen¬ 
erated  by  this  method  will  be  called  transformation  noise. 

If  the  input  v  has  a  multivariate  Gaussian  distribution,  this  transfor¬ 
mation  can  be  modeled  as  shown  in  Fig.  2  where  z  is  an  iid  N(0,1)  random 
vector,  and  L  is  a  linear  operator.  It  is  well  known  that  any  symmetric 
matrix  R  can  be  factored  by  Crout  resolution  [  12]  to  give  the  form: 

R  =  LLr 

where  L  is  an  unique  lower  triangular  (hence  causal)  matrix.  This 
corresponds  to  spectral  factorization  in  continuous  time  which  is  used  to 
solve  the  Weiner-Hopf  equation.  The  linear  filter  L  in  Fig.  2  can  be  there¬ 
fore  chosen  to  produce  a  desired  covariance  in  v: 

E\v vT\  =  £'f(Lz)(Lz)ri  =  L  E\ Lr  =  LLf  =  R 
This  transformation  model  allows  selection  of  the  noise  marginals  but 


Fig.  1.  General  transformation  noise. 


Fig.  2.  Transformation  noise  from  a  Gaussian  background. 


-89- 


increases  the  complexity  of  calculating  the  output  dependency.  The 
advantages  of  the  model  alluded  to  above  are  that  the  only  knowledge  of 
the  source  needed  for  its  complete  statistical  specification  are  the  non- 
Gaussian  marginal  density  and  the  mxm  covariance  matrix. 


DETECTION  PROBLEM 

The  detection  problem  considered  is  formulated  as  a  hypothesis  H0 
and  an  alternative  H\. 

Ho-  x  =  n 
Hi-  x  =  n  +  s 

Under  H0,  the  length  m  observation  vector  x  consists  of  m -dependent 
noise  n  only  and  under  Hx  it  consists  of  signal  s  with  unknown  amplitude 
t?  plus  noise.  It  is  assumed  that  the  signal  is  known  and  that  the  noise 
has  an  m -dimensional  density  /  (n).  A  detector  is  represented  as  a  func¬ 
tional  'k(x)  operating  on  the  observation  x  and  this  scalar  valued  test 
statistic  is  compared  to  a  threshold  to  decide  for  H0  or  Hx  (see  Fig. 3). 


As  the  signal  amplitude  approaches  zero,  a  frequently  used  measure 
of  detector  performance  is  its  efficacy: 


J  W  =  lim 


a-t? 


■£VHx) 


Vor^Cx) 

This  is  a  measure  of  the  asymptotic  performance  of  the  detector  and  is 
maximized  to  produce  the  locally  optimal  test. 


Throughout  this  discussion  we  assume  that  b^th  the  family  of  densi¬ 
ties  /  (x,t?)  and  the  test  statistic  'k(x)  satisfy  certain  regularity  cond. 
tions.  Let  the  test  statistic  have  expected  value 
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£«*(x)  =  /*(x)/  (x.T?)cdM 

To  be  able  to  move  the  derivative  inside  the  expectation  operator  the 
statistic  'J'(x)  is  required  to  be  regular  at  iJq;  that  is  for  all  in  an  open 
neighborhood  of 

£V  *(x)  =  *(x)/  (x,^)dfj.  =  fy Kx)/  '(x,4)dn 

We  also  desire  that  *(x)=l  be  regular: 

/  (xtf)dM  -  fS  '(x,,d)d/4  =  0 

To  satisfy  these  conditions  restrictions  must  be  made  on  the  test  statistic 
SKx)  or  on  the  family  of  densities  /  (x,i3).  Rather  than  overly  limit  the 
class  of  test  statistics,  restrictions  are  applied  primarily  to  the  family  of 
densities.  It  can  be  shown  [13]  that  all  real-valued  test  statistics  with 
finite  second  moment  are  regular  in  the  neighborhood  of  i>0  if  the  family 
of  densities  /  are  smooth  (in  Pitman’s  sense)  at  iS0.  For  family  of  densi¬ 
ties  to  be  smooth  at  i>0 

(i)  /  has  a  iS  derivative  / '  at  almost  every  x  for  each  value  of  U  in  an 
open  neighborhood  of  tJ0. 

(ii)  (/')2//  is  integrable  and  f  {J  ')2/ /  dfj.  <  <»  (finite  Fisher’s  Informa¬ 
tion). 

It  is  well  known  [13]  that  under  the  above  regularity  conditions  the 
function  that  maximizes  efficacy,  the  locally  optimal  test  statistic  ^(x), 
is  the  derivative  of  the  log  likelihood  function  at  t?  =  0: 

*te(x)  =  Um^-log[/(x,i?)] 

In  particular,  for  this  detection  problem  (a  shift  in  mean): 
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*to(*)  = 


zZIM. 


/(*) 


(i) 


where  V/  is  the  gradient  of  / .  The  efficacy  of  this  test  is  given  by 


J(*io)  =  Eo(H) 

In  the  independent  and  identically  distributed  (iid)  case,  the  mul¬ 
tivariate  density  is  the  m-fold  product  of  the  noise  marginal  /i(nt).  Eq. 


(1)  reduces  to  a  ZNL  followed  by  a  correlator,  a  form  considered  exten¬ 
sively  in  the  literature: 


m  f  i'(xj)  m 

+*(*)  =  2W*.) 

<■1  /  t*l 

The  ZNL  gb  is  the  locally  optimal  nonlinearity  for  iid  noise,  given  by 


0to(*i)  =  “  7^**)  (2) 

Ji 

Given  a  specific,  closed  form  multivariate  density  of  the  type  con¬ 
sidered  in  Section  I,  the  locally  optimal  detector  can  be  found  from  Eq. 
(1).  In  the  case  of  the  multivariate  Gaussian  density: 

/ia)s  w&mexfi-nTar,B/z) 

where  R  is  the  covariance  matrix,  the  resulting  detector  is  the  matched 
filter: 

(x)  =  ■rR”1x 

As  a  non-Gaussian  example,  consider  a  bivariate,  elliptically  symmetric 
density  with  Laplace  marginals  [8]: 

/<»>= 

The  resulting  detector  is 


*to(*)  = 


2 

xfiFIx 


JC1(V2xrR~tx) 

#T0(v2x7’R-1x) 


(irir,1x) 


where  Kq  and  Kx  are  modified  Bessel  functions.  The  test  statistic  is  the 
product  of  the  outputs  of  a  matched  filter  and  a  nonlinearity  with 
memory. 

As  in  the  example  above,  closed  form  multivariate  densities  typically 
yield  detectors  which  include  nonlinearities  with  memory,  making  these 
detectors  difficult  to  implement.  Employing  the  transformation  noise 
model  (with  Gaussian  background)  simplifies  the  resulting  detector  to  a 
combination  of  ZNL's  and  a  linear  filter. 

TRANSFORMATION  NOISE  DETECTOR 

As  described  in  the  Introduction,  the  m -dimensional  noise  n  is  gen¬ 
erated  by  passing  an  identically  distributed  noise  sequence  v  with  a 
known  multivariate  density  *p(i/)  and  marginals  through  an  inverti¬ 

ble  ZNL  as  shown  in  Fig.  1.  The  output  noise  n  has  a  multivariate  density 
f  (n)  with  a  dependency  structure  and  marginals  fifa)  determined  by  ip 
and  g.  It  is  assumed  that  g  is  twice  differentiable  almost  everywhere  and 
that  both  n *  and  i \  have  invertible  cdf’s.  The  density  of  n  is  found  with  a 
change  of  variables: 

/(n)  =  se[p(n)]fl  Iff' 00 1 

<■1 

where 

9( n)  =  [  f  (»i).  gM . ?(nm)]r 

The  locally  optimal  detector  is  found  from  Eq.  (1)  to  be 

♦to  (*)  =  -  9  (*)]  9'  (*)  +  £r<x)  » 

y  9 

A  block  diagram  of  ♦j,  is  given  in  Fig.  4  where  the  symbol  ®  is  an  element 


by  element  vector  multiplication  and  0  is  a  vector  dot  product.  Hie  sys¬ 
tem  consists  of  the  locally  optimal  nonlinearity  with  memory  (for  signal 
in  v  distributed  noise),  three  ZNL’s  and  a  correlator  followed  by  a  thres¬ 
hold  comparator. 

The  transformation  ZNL  g  can  be  found  by  equating  the  marginal 
cdf’s  of  n  and  v  (^(nj  and  $1(1/*)  respectively): 


9(*i)  -  *r'[Fi(Xi)] 

The  nonlinearity  g'  is  found  by  differentiating  Eq.  (3) 


and  likewise 


S  i(g») 


(3) 

(4) 


-  £<Xi)  =  -  +  “fo(*»)]  ff'(*i) 

9  J  1  Vi 

At  values  where  the  denominator  of  these  expressions  equals  zero, 
the  quantity  is  assumed  to  equal  infinity.  This  is  acceptable  since  an 
observation  in  a  region  of  zero  probability  of  the  noise  process  indicates 
the  sure  presence  of  the  signal. 

When  v  is  a  multivariate  Gaussian  random  vector  with  unit  variance 
and  covariance  matrix  R,  this  third  nonlinearity  becomes 

-  5r<*i)  =  "  7HO  +  y'(*i)0(*i) 

9  J 1 

The  locally  optimal  nonlinearity,  with  memory,  is  a  linear  filter 

-  ^x)  =  R-1x 

Therefore,  the  system  can  be  reconfigured  as  shown  in  Fig.  5.  The  third 


ZNL  g{e  is  the  locally  optimal  nonlinearity  for  signal  in  iid,  / 1  distributed 
noise  as  given  in  Eq.  (2).  Thus,  under  the  assumption  of  transformed  unit 
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variance  Gaussian  noise  (Fig.  2),  the  system  consists  of  one  linear  filter 
and  three  ZNL’s,  one  of  which  is  the  nonlinearity  occurring  in  the  iid 
locally  optimal  detector. 

Under  the  assumed  noise  model,  the  detector  design  requires 
knowledge  of  g ,  the  nonlinear  transformation,  and  Ry,  the  covariance 
matrix  of  the  background  Gaussian  vector  v.  In  practice,  however,  the 
available  information  is  often  the  noise  marginal  density  and  the  noise 
covariance  matrix  Rn  The  transformation  nonlinearity  g  is  found  from 
the  marginal  cdf  with  Eq.(3).  Computing  Ry  from  in  general  requires 
numerical  integration  (Appendix  A). 


TRANSFORMATION  EXAMPLES 

Two  examples  of  the  proposed  detector  are  considered  in  this  sec¬ 
tion.  The  first  example  employs  the  Laplace  density  as  the  noise  marginal 
and  the  second  example  has  Pearson  Type  VH  marginals.  For  both  of 
these  examples,  Eqs.  (3), (4)  and  (2)  yield  the  ZNL's  g,  g’  and  respec¬ 
tively.  Covariance  mapping  described  in  Appendix  A  relates  the  back¬ 
ground  and  noise  covariance  matrices,  Ry  and  R*. 

The  Asymptotic  Relative  Efficiency  (ARE)  is  often  used  to  compare 
the  performance  of  two  detectors.  It  is  defined  as  the  ratio  of  the 
number  of  samples  needed  to  achieve  the  same  level  and  power  of  the 
two  detectors  as  the  number  of  samples  goes  to  infinity.  With  the  above 
regularity  conditions  and  the  Pitman-Noether  theorem,  the  ARE  is  equal 
to  the  ratio  of  the  efficacies 


ARE  i,2 


■Wx) 
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An  analytic  evaluation  of  the  efficacy  of  these  detectors  is  generally 
intractable  and  depends  on  the  signal  a.  As  a  result,  the  operation  of  the 
optimal  and  several  suboptimal  detectors  was  simulated  to  evaluate  the 
performance  in  two  multivariate  transformation  noise  environments.  The 
simulated  structures  are  numbered  as  follows: 

1.  Locally  optimal  system  (Fig.  5) 

2.  Locally  optimal  test  for  iid  noise  (pto) 

3.  Matched  filter  paralleled  by  g and  the  values  summed 

4.  Matched  filter 

Detector  3  is  included  to  determine  the  effects  of  removing  the  vector 
multiplication  in  the  optimal  detector,  significantly  simplifying  its  struc¬ 
ture.  A  Monte  Carlo  simulation  was  employed  to  compute  the  efficacy  of 
the  four  detectors  above.  These  results  were  then  used  to  compute  the 
ARE  for  each  scheme  as  compared  to  the  matched  filter. 

The  matched  filter  has  efficacy  s^R-1*.  In  a  bivariate  noise  environ¬ 
ment,  with  correlation  coefficient  p,  the  efficacy  of  the  matched  filter  is 
maximized  by  s j  =  -s2  when p  at  0  and  by  Sj  =  s2  when p  <  0.  Conversely, 
it  is  minimized  by  Sj  =  s2  for  p  a  0  and  by  Sj  =  -s2  for  p  <  0.  Since  all 
four  systems  are  symmetric  in  sx  and  s2,  the  condition  st  =  s2  maximizes 
while  sx  =  -s2  minimizes  efficacy  for  p  s  0  (the  extrema  are  reversed  for 
p  as  0).  A  constant  signal  with  unit  power  is  used  in  the  evaluation  of 
bivariate  detectors  in  this  chapter,  and  therefore,  the  simulation  yields 
minimum  and  maximum  efficacies  for  p  &  0  and  p  <  0  respectively. 

The  bivariate  simulation  shows  ARE  for  -1  <  p  <  1.  However,  for  m- 


dependent  noise,  this  entire  range  may  not  be  of  importance.  Since  all 
terms  of  the  m -dependent  autocorrelation  sequence  beyond  the  first  m 
vanish  anti  the  covariance  matrix  must  be  positive  definite,  bounds  can 
be  placed  on  the  correlation  sequence  [14].  For  the  bivariate  case 
(m  =  2),  the  bounds  on  p  are  \p\  with  the  result  that  the  values  of 
ARE  where  1p|  are  perhaps  of  the  most  interest.  Although  the 
efficacies  are  maximized  or  minimized  by  the  selection  of  the  signal,  this 
does  not  appear  in  the  plots  of  ARE.  This  results  because  the  matched 
filter  is  more  sensitive  than  many  other  detectors  to  signal  selection. 

For  multivariate  simulations  of  length  m,  a  triangular  correlation 

v 

function  sampled  at  equal  intervals  is  assumed: 

1  -  \i/m  |  |i  |  «  m 
=  |i  j  >  m 

This  triangular  function  approximates  the  type  of  correlation  often  seen 
in  highly  correlated  noise  sequences.  For  these  examples,  the  signal  to 
maximize  the  efficacies  is  no  longer  obvious  due  to  the  nonlinearities 
present  in  the  detectors.  Rather  than  find  the  signal  to  maximize  the 
efficacy,  simulations  are  presented  with  a  constant  (s*  =  k)  and  an  oscil¬ 
lating  (»  =  \k •  •  {  )  signal.  These  were  chosen  because  they  yield 
nearly  the  worst  and  best  performance  from  the  matched  filter  respec¬ 
tively  for  the  correlation  sequence  above. 
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1.  Laplace  Results 

Impulsive  noise  environments  are  often  modeled  by  densities  whose 
tails  are  heavier  than  the  Gaussian  [15,16].  The  Laplace  density  is  chosen 
as  a  first  example  since  it  is  a  frequently  used  model  of  non-Gaussian, 
impulsive  noise  [17].  For  the  marginal  density 


fiM  ~  |-exp(— alnj) 


Eqs.  (3),  (4)  and  (2)  yield  the  three  detector  ZNL’s 


p[l  +  sgnfa)  [l-exp(-a|xi|)]J 


g'(xi')=  Si:^-exp(gz(xi)/2-ajxil) 


ffto (*t)  =  a  sgn  (xj 

where  is  the  inverse  normal  cdf.  For  unit  power  noise  a  =  >/2  and 


correlation  coefficient  mapping  shows  that  p  *  /ot.  Figs.  6,  7  and  8  con¬ 
tain  the  three  nonlinearities  g,  g'  and  pto.  The  resulting  ARE’s  for  the 
bivariate  case  are  shown  in  Fig.  9,  and  for  the  multivariate  case  in  Figs. 


10  and  11. 


2.  Pearson  Type  VII  Results 

The  Pearson  VII  density  is  selected  as  a  second  example  because  it 
car.  be  made  arbitrarily  close  to  Gaussian  by  changing  one  of  its  parame¬ 
ters  [11].  Here  the  marginal  is 

/iW  =  K  [l  + 

As  the  density  approaches  the  Gaussian.  For  this  example,  let  a  =  7. 
This  results  in  a  density  with  a  nearly  Gaussian  body  and  algebraically 
decaying  tails.  The  three  nonlinearities  in  the  detector  are  found  from 
Eqs.  (3),  (4)  and  (2)  to  be 

g(xi)  ~ 

£’(*»)  =  X  >/2tt  [l+(xi/a)2]7exp(g2(zi)/2) 

9u>(*i)  =  14*j/  (a2  +  Zj2) 

where  is  the  Pearson  VII  cdf.  These  ZNL’s  are  depicted  in  Figs.  12,  13 
and  14.  In  Fig.  15,  the  ARE’s  generated  by  simulation  for  bivariate  noise 
are  plotted.  The  multivariate  simulation  results  are  depicted  in  Figs.  16 
and  17  for  constant  and  alternating  signals  respectively. 


1.4 


with  a  constant  signal. 
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Fig.  16.  ARE  of  Pearson  VII  detectors  vs.  detector  length  m  for  triangular 
correlation  and  a  constant  signal. 


Fig.  17.  ARE  of  Pearson  VII  detectors  vs.  detector  length  m  for  triangular 
correlation  and  an  alternating  signal. 


MOVING  AVERAGE  (UA)  NOISE  DETECTOR 

Modeling  the  noise  process  as  in  Fig.  2  suggests  the  reversed  model 
shown  in  Fig.  18.  The  input  z,  iid  N(0,1),  is  passed  through  the  ZNL  p-1  to 
produce  the  iid  vector  7  with  non-Gaussian  marginals  <Pi  and  variance  o2. 
Passing  7  through  a  linear  filter  L  of  length  m  produces  the  sequence  n 
with  a  dependence  structure  generated  by  L  This  model  is  typically 
called  a  moving  average  process  (MA)  of  order  m-1  [14]. 

As  in  the  Introduction,  Crout  factorization  can  be  used  to  solve  for  an 
unique  lower  triangular  matrix  L>  given  R,  since  the  covariance  matrix  of 
n  is  given  by 

R*  =  E\nnT\  =  E\(Ly)(L7)T\  =  LE[yyT\  LT  =  o*UJ 
where  a2  is  the  variance  of  the  7*.  This  MA  model  has  a  more  straightfor¬ 
ward  dependence  structure  than  the  transformation  noise,  but  solving  for 
the  marginals  is  more  difficult.  Note  that  these  two  schemes  are 
equivalent  only  when  p-1  is  Unear  (n  is  multivariate  Gaussian). 

Denoting  the  7  marginal  as  ?i(7»)»  the  density  of  the  MA  noise  n  is 
found  to  be 

f{n)  =  |L-,I 

i*l 

where  <pi  is  assumed  continuously  differentiable  and  strictly  positive.  The 
locaUy  optimal  detector  ^  (x)  is  found  from  Eq.  (1)  to  be 

=  (L-1s)rpto  (L-1x) 

where  pto  is  the  ZNL  in  the  iid  locally  optimal  detector  for  the  <p\  margi- 


A  block  diagram  of  this  detector  is  given  in  Fig.  19.  The  system  is  a  ZNL 
embedded  in  a  matched  filter.  Note  that  for  n  iid,  L  is  an  identity  matrix, 
and  the  resulting  detector  is  the  iid  locally  optimal  detector  sT0ie(*)-  For 
the  Gaussian  case  [p"1(x)  =  x]  the  result  is  the  matched  filter  srR-1x. 

The  efficacy  of  the  locally  optimal  test  statistic  is 

where  E0  is  the  expectation  under  Hq.  This  can  be  calculated  for  the 
moving  average  model: 

J  (*)  =  (L_1a) T  E \gio  (L ~'x)g£ (L_,x)  {  (L-1s) 
where  is  the  ZNL  embedded  in  the  detector.  The  outer  product  under 

the  expectation  can  be  simplified  since  L_1x  =  7  is  an  iid  random  vector. 

The  nonlinearity  grto  has  expected  value  zero  for  any  locally  optimal 

detector  since 

£fsi.t  »-/*/« 

The  outer  product  reduces  to  a  diagonal  matrix  with  diagonal  elements: 

E0\g&(x)\  =  I0 

where  /0  is  Fisher’s  information  for  the  marginal  density  f  v  The  efficacy 
of  the  system  becomes 

•rR-‘« 

and  the  ARE  as  compared  to  the  matched  filter  is 

AREk.mf  -lo/a2 

Fisher's  information  is  known  to  have  a  minimum  of  o2  for  the  Gaussian 
density;  hence,  the  ARE  is  always  greater  than  or  equal  to  unity  and  the 
locally  optimal  detector  performs  at  least  as  well  as  the  matched  filter. 


CONCLUSIONS 


Given  a  completely  specified  multivariate  density,  methods  of  finding 
optimal  detectors  are  well  known.  If  the  noise  is  non-iid  or  non-Gaussian, 
the  complete  statistical  description,  the  m -dimensional  density,  may  be 
an  unreasonable  amount  of  information  to  expect  in  practice.  The 
transformation  and  moving  average  density  models,  however,  are  charac¬ 
terized  solely  by  the  marginal  densities  and  the  covariance  matrices. 
Also,  the  use  of  moving  average  and  transformation  noise  models  as 
structures  for  a  non-Gaussian  noise  environment  simplifies  the  locally 
optimal  detector  structure  since  no  nonlinearities  with  memory  are 
required. 

Three  of  the  four  detector  structures  considered  in  the  simulations 
are  of  particular  importance.  The  matched  filter  uses  the  covariance 
matrix  only  and  is  optimal  when  the  density  is  Gaussian.  The  iid  locally 
optimal  detector  employs  the  marginal  density  of  the  noise  and  is 
optimal  if  the  noise  is  independent.  The  multivariate  transformation 
noise  detector  uses  both  the  correlation  and  the  marginals,  and  is 
optimal  or  nearly  optimal  when  the  noise  is  described  well  by  the 
transformation  noise  model. 

In  the  bivariate  case  the  effects  of  dependence  upon  performance  as 
indicated  in  the  simulations  are  minimal  (particularly  for  |p|^J£).  The  iid 
locally  optimal  detector  performed  nearly  as  well  as  the  multivariate 
locally  optimal  detector.  Both  detectors  performed  better  than  the 
linear  detector  indicating  that  in  fact  the  form  of  the  marginal  is  more 
important  than  the  dependence  structure. 
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For  longer  correlation  sequences  (m>2),  the  dependence  structure 
becomes  increasingly  more  important.  In  the  simulations,  the  ARE  of  the 
iid  locally  optimal  detector  falls  well  below  that  of  the  optimal  detector. 
When  the  signal  is  chosen  to  nearly  optimize  the  matched  filter  perfor¬ 
mance,  even  the  matched  filter  performs  better  than  the  iid  locally 
optimal  detector.  In  this  case,  the  dependence  structure  and  the  margi¬ 
nals  are  both  important  and  a  detector  structure  should  exploit  both 
when  possible. 

The  moving  average  model  was  introduced  as  a  permutation  of  the 
transformation  formulation.  Simulations  were  not  attempted  because  of 
the  need  to  factor  the  marginal  characteristic  function  to  arrive  at  a  y 
process.  However,  the  detector  structure  itself  is  simple  and  suggests  its 
use  as  a  general  suboptimal  structure  which  can  be  optimized  numeri¬ 
cally. 
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APPENDEX  A  -  Computation  of  Rv 

Implementation  of  the  transformation  noise  detector  requires 
knowledge  of  the  covariance  matrix  of  the  background  Gaussian  process. 
Assuming  that  the  input  and  output  processes  have  unit  power,  this  prob 
lem  reduces  to  finding  a  mapping  between  the  noise  correlation 
coefficient  Pn  and  the  background  process  correlation  pv  [18].  The  out¬ 
put  correlation  coefficient  is  given  by 

Pn(i)  =  EfajUj+il  = 
where  p-1  is  defined  to  be 

Since  the  Uj  and  uJ+i  are  unit  normal  random  variables,  with  correlation 
coefficient  p  =  p„(i),  the  expression  for  pn(i)  becomes 


Pn(0  =  f  f  9"1(x)g~1(y) 


•exp 


_  x2-Z oxy+y2 


2(1 -P2) 


dx  dy  (Al) 


2Wi-p2 

Solution  of  this  integral  expression,  usually  numerically,  provides  the 
required  mapping  of  p„  to  pn 

In  [19],  Wise  and  Thomas  express  the  joint  normal  density  as  a  series 
expansion,  Mehler’s  formula: 

jv(o,o,u,p)  =  ^expHg2+y8)/2j  tQP 

where  Hk  is  the  fc-th  Hermite  polynomial.  The  nonlinearity  p-1  can  be 
represented  by  its  Hermite  expansion: 


with  coefficients  bk  given  by 


bk  =  fff'Hx)  ^j-exp(-i2/2) 


Hk(x) 

VjfcT 


dx 


Employing  the  orthogonality  relationship  of  the  Hermite  polynomials: 


2 #*(xWx)exP(“  xZ/2)  «**  ~  6:k 

the  integral  in  Eq.(A.l  )  reduces  to 

A,(i)=  £ 

k  =0 

In  the  noise  examples  in  this  chapter,  the  marginal  density  is 
assumed  to  be  symmetric;  hence,  both  g  and  ^-1  are  odd  symmetric. 
The  even  Hermite  expansion  coefficients  (6 2*.  k=0,l,...)  are  zero  and 
pn(i)  is  an  increasing,  odd  symmetric  function  of  p„(i).  Therefore  in 
practice,  the  values  of  p„  can  be  interpolated  from  a  table  of  p„  versus  p* . 
Also,  as  an  added  note,  the  Gaussian  process  has  a  correlation  coefficient 
that  is  always  larger  in  modulus  than  the  output  correlation: 

IpJ  *  I Pn  I 


APPENDIX  B  -  Estimation  of  Transformation  Nonlinearity. 

Suppose  a  particular  multivariate  non-Gaussian  noise  process  is  well 
described  by  the  transformation  model  with  Gaussian  pre-transformation 
noise.  Then  it  is  assumed  that  the  length  m  non-Gaussian  noise  vector  n 
with  multivariate  density  /m(n)  can  be  produced  by  passing  Gaussian 
noise  v  with  denity  <pm  (u)  through  an  invertible  zero  memory  nonlinearity 
(ZNL)  as  shown  in  Fig  31.  It  is  assumed  that  both  <pm{y)  and  /m(n)  each 
have  identical  marginals  (?( i/*)  and  /  (rq)  respectively). 

If  the  marginals  <p  and  /  are  both  known,  then  g  can  be  found  by 
equating  marginal  cdfs  and  the  problem  is  solved.  Frequently  in  prac¬ 
tice,  /  may  not  be  known  exactly,  and  an  approximation  to  this  ZNL  with 
easily  estimated  parameters  may  be  desired.  For  instance,  the  locally 
optimal  detector  for  transformation  noise  derived  in  this  chapter 
requires  g  given  only  the  observed  post-transformation,  non-Gaussian 
noise  n.  Under  certain  regularity  conditions  on  the  f,  the  Cornish-Fisher 
expansion  can  be  used  to  generate  an  approximate  transformation  of  the 
form  [11,20]: 

v  -  g{n)  =  a0  +  ajTi  +  a2n2  +  •  •  • 

Certain  regularity  conditions  on  /  are  required.  The  Edgeworth 
expansion  (and  therefore  the  cumulants)  of  /  should  exist: 


\  . _  (k2  ~  ct2)/?2  k$D3  K4D 4  . 

/(n)  =  exp|-(«i  -n)D  +  - - - —+  — - -  •  •  Mi) 

where  <p(x)  is  Nin.o2),  D  is  the  differential  operator  and  Ky.  is  the  r 01 


eumulant  of  /.  The  rth  cumulant  is  required  to  be  of  order  0(  TJ1“r  ), 
where  is  chosen  so  that  /  approaches  Gaussian  as  for  example, 


might  be  sample  size.  In  general  these  conditions  are  not  too  severe,  and 
are  satisfied  by  most  common  densities. 


=  <3  =  *5  =  0  and  kz  -  1  k4  =  3  «6  =  30 

From  Eq.(Bl)  the  approximate  transformation  is 


g(n)=  201^.1^3+ 

'  120  24  384 


The  exact  transformation  is 

v  -  gin )  -  $_,[l  -  ^exp(-V2n)] 

where  $  is  the  unit  normal  cdf.  The  exact  and  approximate  transforma 
tions  are  plotted  in  Fig  Bl. 

The  second  example  is  the  Pearson  VII: 

fix)  =tf(l  -(x/o)2)"* 

As  in  the  examples  earlier  in  this  chapter,  let  m  =  7.  Then  o2  =  -11  an< 
the  cumulants  are 

Kj  =  k3  =  k 5  =  0  and  =  1  ka  =  2/  3  **  =  13 

Then  from  Eq.  Bl  the  approximate  transformation  is: 


9  in)  +  0 

The  exact  and  approximate  transformation  is  plotted  in  Fig.  B2. 
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CHAPTER  6  -  NOISE  MODELS  FOR  DETECTION! 


INTRODUCTION 

Problems  in  binary  detection  involve  choosing  between  two  statistical 
environments,  described  as  an  hypothesis  Hq  and  an  alternative  Hi 

Hq.  Pq(t) 

Hi  P,(x) 

This  decision  is  based  on  a  statistic  of  an  observation  sequence  z  of 
length  m .  Various  functionals  of  the  vector  z  are  traditionally  employed 
to  reduce  the  data  to  a  scalar  statistic  easing  the  resulting  detection  pro¬ 
cess.  This  test  statistic  is  compared  to  a  threshold  T  to  decide  for  H0  or 
H i  (Fig.  1).  For  the  criteria  of  detector  optimality  often  considered  [l] 
(Neyman-Pearson.  Bayes,  probability  of  error)  the  Likelihood  Ratio  (LR) 
is  known  to  be  the  optimal  statistic  of  the  observation  x.  A  second  detec¬ 
tor  structure  often  used  is  the  Locally  Optimal  or  small  signal  detector 
(LO).  Both  the  LR  and  LO  detectors  are  considered  here  for  several  mul¬ 
tivariate  noise  models. 

The  class  of  problems  considered  here  is  that  of  a  deterministic  sig¬ 
nal  in  additive  noise.  Hq  and  Hx  become 

Hq.  x  =  n 

H\.  z  =  n  +  •#»  iJ  >  0 

Under  Hq  the  observation  consists  of  noise  n  with  density  /( n),  and 
under  Hx  the  observation  consists  of  a  signal  s  with  amplitude  if  plus 
noise  n. 

This  chapter  was  co-authored  with  Peter  F.  Swaszek,  and  also  appears  in  his  PhD 
dissertation  Robust  Quantisation,  Victor  Quantisation  and  Dstsction,  Princeton 
University,  October  1962. 
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Fig.  1  -  General  detector  structure. 


The  likelihood  ratio  is  given  by 


LR(X)  =  L^xl=  lA*r$.H , 
W  /oM  /(*) 


(1) 


When  the  amplitude  constant  is  small  and  unknown,  the  locally  optimal 
test  statistic 


* 


(2) 


is  frequently  employed  [2]. 

Throughout  this  chapter  a  monotone  function  of  LR  or  LO  may  be 
used  in  place  of  LR  or  LO  when  it  simplifies  a  detector  structure.  This 
can  be  done  without  loss  of  generality  since  taking  a  monotonic  function 
of  both  the  test  statistic  and  the  threshold  value  does  not  effect  the  per¬ 
formance  of  the  test.  The  log  function  is  often  used  since  the  test  statis¬ 
tic  for  an  independent  noise  process  reduces  to  a  sum  of  zero  memory 
nonlinear  functions  of  the  observations. 


MULTIVARIATE  DENSITIES 

In  this  section  several  methods  of  generating  families  of  multivariate 
noise  densities  are  considered.  Methods  that  produce  large  families  of 
densities  are  of  particular  interest  since  they  are  more  variable  and  may 
provide  a  closer  fit  to  the  actual  noise.  For  each  family  of  densities,  LR 
and  LO  detectors  are  derived  and  discussed.  It  should  be  noted  that 
some  noise  models  produce  structures  which  are  more  practical  than 
others,  and  the  practicality  of  each  structure  is  considered. 

CLOSED  FORMS 

There  are  a  number  of  well  known  closed  form  multivariate  densities 
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[3].  In  general  these  are  named  for  their  marginals,  which  are  often  com¬ 
mon  univariate  densities.  Unfortunately,  the  marginal  densities  do  not 
uniquely  determine  a  multivariate  density,  and  in  many  cases  it  is  not 
even  possible  to  make  the  conditional  densities  have  the  same  form  as 
the  marginals.  Although  closed  form  multivariate  densities  may  result 
from  a  particular  model,  more  often  they  are  the  result  of  extending  a 
characteristic  of  the  univariate  density.  While  they  have  the  advantage  of 
tractability,  there  is  usually  very  little  control  over  the  dependency 
structure.  LR  and  LO  detectors  as  a  rule  contain  nonlinearities  with 
memory  which  may  prove  difficult  to  implement. 

In  the  case  of  the  multivariate  Gaussian  density 


/<n)=(i^Wexp 

where  R  is  the  m  xm  covariance  matrix,  the  resulting  detectors  {LO  and 
LR)  are  both  the  matched  filter: 

LR{x)  =  LO{x)  =  srR_,x 
The  multivariate  Cauchy  density  [4]: 

K 


f  (°)  -  - - : - i37Z~ 

cz  +  n^R-^n 


yields  the  test  statistics: 

LR(x)  = 
and 


c2  +  xTR"1* 


c2  +  (x-T?s)rR"1(x-'ds) 


LO(x)  = 


sTvr 


i. 


c2  +  xrR-Jx 
Both  contain  non-linearities  with  memory. 


DIFFERENTIAL  EQUATIONS 


One  class  of  noise  models  is  of  particular  interest  in  LO  detection. 
The  detector  nonlinearity  (x,a)  parameterized  by  a  is  chosen  for  trac- 
tability.  Eq.(2)  can  be  viewed  as  a  vector  differential  equation  and  solved 
to  give  a  family  of  solution  densities  /^x, a).  A  member  density  of  this 
family  can  then  be  fitted  to  the  noise  by  choosing  a.  This  estimate  of  a 
can  be  inserted  in  g ,  and  the  result  is  a  fitted  LO  nonlinearity. 

This  method  has  been  applied  when  is  assumed  to  be  a  rational 
function  [5]  The  resulting  family  of  densities  is  the  Pearson  family  which 
is  particularly  well  adapted  to  using  the  method  of  moments  to  estimate 

a. 

This  approach  has  the  advantage  of  producing  an  adaptive  detection 
system,  but  there  is  no  guarantee  that  any  member  of  the  family  of  solu¬ 
tion  densities  will  be  a  good  fit  for  a  given  noise  process.  The  detector 
structure  is  pictured  in  Fig.  2. 

ELLIPTICALLY  SYMMETRIC 

Another  way  to  characterize  multivariate  densities  is  to  require  that 
the  pdf  have  contours  of  constant  height  which  are  ellipsoids  in  m -space. 
These  elliptically  symmetric  densities  can  be  generated  by  replacing  the 
independent  variable  of  a  univariate  density,  say  /  i(n),  with  a  quadratic 
form  nrR-1n.  The  resulting  density  has  the  form: 

/(n)  =  */,[v?7r^) 

where  A'  is  a  scaling  constant  and  R  is  similar  to  a  correlation  matrix. 
However,  the  resulting  multivariate  density  does  not  in  general  have  as 


itial  equation  LO  detector 
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its  marginals  the  univariate  density  /i(n). 

There  is  a  method  of  generating  elliptically  symmetric  densities  with 
a  specific  marginal  /i(n).  Assume  the  marginal  has  characteristic  func¬ 
tion  (p and  let  the  m-dimensional  characteristic  function  be 

Taking  the  inverse  transform  yields  the  m-dimensional  density  /( n). 
Care  must  be  taken  to  ensure  that  /  (n)  integrates  to  unity  and  is  posi¬ 
tive  for  all  n.  Thus  the  marginals  can  be  controlled  but  the  dependency 
structure  of  n  is  not  related  to  R  in  a  simple  or  obvious  way. 

Since  the  argument  of  /  is  a  quadratic  form,  the  detector  structures 
can  be  written  as 


and 


LR(x)  = 


/  !  (v (x— iJs)  rR-1  (x—tf  s)] 


LO(x)  =  srR-1x 

Block  diagrams  of  these  systems  appear  in  Fig.  3. 

\ 

As  a  non-Gaussian  example,  consider  an  elliptically  symmetric  den¬ 
sity  with  Laplace  marginals  [6]: 


/(n)  = 


The  resulting  detectors  are 


_  A’0[V2(x^s)rR-1(x-^s) 
LJt(x)  -  1  /  ;  *- 

A'o[V2xfR_1xJ 


and 


where  Kq  and  K\  are  modified  Bessel  functions.  The  test  statistics  both 
involve  nonlinearities  with  memory. 

SERIES 

It  is  well  known  that  under  certain  regularity  conditions  a  density 
/  (n)  with  marginals  fi(rii)  can  be  expanded  in  series  form  [7]: 

/(*)=  ff/i(*0  S  •  ■  S 

i=l  j  1=0  jm-0  *=1 

where  the  ^kjk  are  sequences  of  orthonormal  functions  with  generating 
functions  fj .  The  coefficients  are  given  by: 

cU~Jm  =  //(“)  dn 

Series  forms  have  a  certain  theoretical  elegance  and  can  be  used  to 
derive  general  properties  of  densities.  However,  in  practice,  it  is  usually 
necessary  to  truncate  the  series,  resulting  in  a  poor  representation  and 
possibly  even  negative  values  in  the  tail  regions  of  the  pdf  [8].  Even  if 
this  difficulty  is  over-looked,  detector  nonlinearities  for  truncated  series 
are  rational  forms  that  are  poorly  behaved  for  observations  falling  in  the 
tail  regions. 

TRANSFORMATION  NOISE 

Another  class  of  multivariate  densities  are  those  generated  by 
transformations  from  other  densities  [8].  Common  univariate  examples 
are  the  log-normal  and  the  Johnson  family.  In  a  multivariate  setting,  this 
method  involves  a  nonlinearity  (possibly  with  memory)  operating  on  a 


noise  sequence  with  a  known  density.  The  intractability  of  this  general 
class  of  transformations  suggests  constraining  interest  to  those  inverti¬ 
ble  transformations  with  zero  memory  (ZNL).  A  block  diagram  of  the  sys¬ 
tem  generating  this  type  of  noise  is  given  in  Fig.  4.  Given  an  identically 
distributed,  m-dependent  background  noise  process  v  with  a  known  den¬ 
sity  ?(▼)  and  marginals  a  ZNL  g  can  be  selected  to  produce  a  noise 

n  with  the  desired  marginals  /i(n<).  If  ^  and  g  are  fixed,  the  depen¬ 
dence  structure  of  n  is  completely  determined  by  the  background  noise 
dependency.  This  lack  of  flexibility  in  choosing  a  dependence  structure 
may  be  a  disadvantage  in  some  cases. 

The  density  of  n  is  found  with  a  change  of  variables: 

/(«0  =  ?[ff(n)l  I!  Iff '(*01 

i=l 

where 


ff(n)  =  [ff(n,).*(n8),  ...,$(0  ]T 

The  transformation  ZNL  g  can  be  found  by  equating  the  marginal 
cdfs  of  n  and  ▼  (/^(n*)  and  $iCiO  respectively): 

fffa)  =  *”1[J’i(n*)]  (3) 

The  LR  detector  is  found  from  Eq.(l): 

=  ,0*  2l^fI+  !.log  9JrGr~ 

which  is  the  p-noise  LR  detector  added  to  a  sum  of  the  observations 
through  a  zero  memory  non-linearity.  The  nonlinearity  g'  is  found  by 
differentiating  Eq.(3) 

ff'(*0  =/i(*t)/  ?j[ff(*i)] 

The  block  diagram  of  this  detector  appears  in  Fig.  5a. 


The  locally  optimal  detector  LO(x )  is  found  from  Eq.(2)  to  be 


LO(x)  =  - 
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A  block  diagram  of  LO(x)  is  also  given  to  Fig.  5b  where  the  symbol  0  is  an 
element  by  element  vector  multiplication  and  0  is  a  vector  dot  product. 
The  system  consists  of  the  locally  optimal  nonlinearity  with  memory  (for 
signal  in  <p  distributed  noise),  three  ZNL's  and  a  correlator  followed  by  a 
threshold  comparator. 

It  can  be  seen  that  if  the  background  process,  <p  is  chosen  to  be  Gaus¬ 
sian  with  correlation  matrix  IV  the  LR  system  reduces  to 

LR(x)  =  +  £  1°6  * 

»»1  /  lV*!!  c  »«1 

which  consists  of  a  matched  Alter  and  zero  memory  nonlinearities  (Fig. 
6a).  This  can  be  considerably  easier  to  realize  than  the  nonlinearities 
with  memory  required  by  other  schemes. 

For  the  LO  detector,  the  third  nonlinearity  is 


-£r(*i)  =  -4m**)  + 

9  J  l 

The  locally  optimal  nonlinearity,  with  memory,  is  a  linear  Alter 

-■^(x)  =  IV1* 

Therefore,  the  LO  system  can  be  reconfigured  as  shown  in  Fig.  6b.  The 
third  ZNL  is  the  locally  optimal  nonlinearity  for  signal  in  iid,  /  j  distri¬ 
buted  noise  as  given  in  Eq.(2).  Thus,  under  the  assumption  of 
transformed  unit  variance  Gaussian  noise,  the  system  consists  of  one 
linear  Alter  and  three  ZNL’s,  one  of  which  is  the  nonlinearity  occurring  in 
the  iid  locally  optimal  detector. 


Under  the  assumfed  noise  model,  the  detector  designs  require 
knowledge  of  the  nonlinear  transformation  g  and  R*.  the  covariance 
matrix  of  the  background  Gaussian  vector  v.  In  practice,  however,  the 
available  information  is  often  the  noise  marginal  density  and  the  noise 
covariance  matrix  K  The  transformation  nonlinearity  g  is  found  from 
the  marginal  cdf  with  Eq.(3).  Computing  R,  from  R*  in  general  requires  a 
numerical  integration  solution  [9,10]. 

If  the  input  ▼  has  a  multivariate  Gaussian  distribution,  this  transfor¬ 
mation  can  be  modeled  as  shown  in  Fig. 7  where  z  is  an  iid  N(0.1)  random 
vector,  and  L  is  a  linear  operator.  It  is  well  known  that  any  symmetric 
matrix  R*  can  be  factored  by  Crout  resolution  [  1 1]  the  give  the  form: 

Rr=LLr 

where  L  is  an  unique  lower  triangular  (hence  causal)  matrix.  The  linear 
filter  L  in  Fig. 7  can  be  therefore  chosen  to  produce  a  desired  covariance 
in  ▼: 

E[wT]  -  £,[(Lz)(Lz)r]  =  L£[zzr]  Lr  -  LLr  =  R„ 

Modeling  noise  in  this  way  suggest?  the  second  model  shown  in  Fig.  8. 
The  input  z,  iid  N(0,1),  is  passed  through  the  ZNL  g~l  t^  produce  the  iid 
vector  w  with  non-Gaussian  marginals  and  variance  o2.  Passing  w  through 
a  linear  filter  L  of  length  m  produces  the  sequence  n  with  a  dependence 
structure  generated  by  L  This  model  is  typically  called  a  Moving  Average 
process  (MA)  of  order  m-1  [12].  As  above,  Crout  factorization  can  be 
used  to  solve  for  an  unique  lower  triangular  matrix  L,  given  R* ,  since  the 
covariance  matrix  of  n  is  given  by 

R*  =  E[jmT]  =  £’[(Lw)(Lw)r]  =  L£[wwr]  L  =  c*Il/ 


The  major  difficulty  in  using  this  model  is  selecting  the  marginal  density 
of  w  which  yields  the  desired  marginal  for  n.  Denoting  the  w  marginal  as 
the  density  of  the  MA  noise  n  is 


£ho(*i)  =  “ 

Vi 

Block  diagrams  of  these  detectors  are  given  in  Fig.  9.  The  LO  system  is  a 
ZNL  embedded  in  a  matched  filter.  Note  that  for  n  iid,  L  is  an  identity 
matrix,  and  the  resulting  LO  detector  is  the  iid  locally  optimal  detector 
Jgu,  (x).  For  the  Gaussian  case  (j7-1(z)=z)  the  result  is  the  matched 
filter  (linear  detector)  s^R^'x. 
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CONCLUSIONS 

This  chapter  presents  several  multivariate  noise  densities  and  their 

associated  LO  and  LR  detector  structures. 

1-  Closed  form  densities  are  tractable  but  are  not  general  in  any  sense, 
and  there  is  little  or  no  control  over  the  dependency  structure. 

2-  The  differential  equation  method  applies  only  in  the  LO  case.  There 
is  a  great  deal  of  control  over  the  functional  form  of  the  detector 
nonlinearity,  but  there  is  no  guarantee  that  the  corresponding  family 
of  solution  densities  provides  a  good  fit  to  a  given  noise  process. 

3-  The  elliptically  symmetric  densities,  although  yielding  detectors  con¬ 
taining  nonlinearities  with  memory,  are  not  difficult  to  implement. 
However,  the  meaning  of  the  matrix  R  in  the  quadratic  form  is 
unclear  and  the  dependence  is  difficult  to  control. 

4-  The  series  forms  have  difficulties  in  implementation  due  to  the 
necessity  of  truncation.  The  generally  slow  convergence  of  series, 
and  the  poor  tail  behavior  usually  noted  when  the  series  is  truncated 
result  in  very  poor  behavior  of  the  detector  nonlinearity  in  the  tail 
region. 

5-  The  transformation  model  allows  selection  of  the  noise  marginals  but 
increases  the  complexity  of  calculating  the  output  dependency.  The 
MA  model  has  a  more  straightforward  dependence  structure  but  solv¬ 
ing  for  the  marginals  is  more  difficult.  Note  that  these  two  schemes 
are  equivalent  only  when  is  linear  (n  is  multivariate  Gaussian). 


1 
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CHAPTER  7  -  FINITE  LENGTH  DISCRETE  HATCHED  FILTERS 

INTRODUCTION 

The  design  and  implementation  of  the  Matched  Filter  (MF)  has 
received  considerable  attention  [4-8].  As  a  detector  it  has  the  advantage 
of  linearity,  and  since  it  is  based  only  on  easily  estimated  2nd  order  noise 
statistics,  the  MF  is  simple  to  optimize.  The  performance  criterion,  the 
Signal-to-Noise  Ratio  (SNR)  is  tractable,  and  intuitively  appealing. 

For  a  fixed  signal  in  discrete  time,  Levinson  [8]  has  presented  a  sim¬ 
ple  and  efficient  algorithm  to  solve  the  MF  equation.  Since  the  MF 
impulse  response  and  the  SNR  are  computed  iteratively,  the  algorithm 
can  be  terminated  when  a  filter  with  desired  performance  is  found. 
Unfortunately,  when  there  is  some  freedom  in  choosing  a  signal,  the 
choice  of  signal  plays  an  important  part  in  optimizing  the  detector. 
Because  the  optimal  signal  of  length  M  is  a  truncated  version  of  the 
optimal  length  Af+1  signal  under  only  very  special  conditions,  the  Levin¬ 
son  algorithm  must  be  repeated  N  times,  and  thus  loses  its  computa¬ 
tional  advantage.  In  this  chapter,  easily  computed  bounds  on  the  perfor¬ 
mance  of  the  MF  as  a  function  of  length  are  found.  Then,  before  any 
attempt  is  made  to  solve  the  MF  equation,  an  estimate  of  the  filter  length 
can  be  found  from  these  bounds. 

DETECTION  PROBLEM 

The  detection  problem  considered  in  this  chapter  is  one  of  finding  a 
linear  detector  that  discriminates  between  an  hypothesis  H0  and  an 
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alternative  Hx.  The  decision  is  based  on  a  discrete  length  N  observation 
vector  z  composed  under  H0  of  noise  n  with  density  /  and  under  Hx  of  a 
known  signal  a  in  noise: 


Hq.  x  =  n 
Hx:  x  =  n  +  a 

The  detector  consists  of  a  real  scalar  test  statistic  r(x),  a  functional  of 
the  observation  z,  which  is  compared  to  a  scalar  threshold  to  decide  for 
or  Hv 

The  criterion  of  detector  optimality  used  in  this  chapter  is  a  SNH 
measure  often  called  the  deflection : 


SNR .  wn  -w?  (1) 

SNR_  Ifcr0  (T)  (1) 

where  E0  and  Ex  are  the  expectation  under  Ho  and  Hx,  and  Var0  is  the 

variance  under  H0. 


It  is  well  known  that  the  log  likelihood  ratio  detector  for  Gaussian 
noise  is  linear,  the  matched  filter.  Since  the  detector  power  is  a  mono¬ 
tone  increasing  function  of  the  SNR  of  T,  the  SNR  is  frequently  used  as  a 
measure  of  detector  performance.  The  SNR,  outside  of  its  intuitive 
appeal,  is  often  justified  by  making  a  Gaussian  assumption  about  n  or 
applying  the  central  limit  theorem  to  T. 


Using  the  MF  as  a  detector  for  non-Gaussian  noise  is  more  difficult  to 
justify.  In  general,  the  likelihood  ratio  detector  maximizes  the  SNR  [l], 
and  by  a  simple  calculus  of  variations  argument,  maximizing  the  SNR  (as 
defined  above)  with  no  restriction  on  the  linearity  of  the  detector  can  be 
shown  to  yield  a  linear  function  of  the  likelihood  ratio.  The  MF  is  the 
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linear  filter  which  maximizes  output  SNR,  but  the  likelihood  ratio  is  gen¬ 
erally  nonlinear.  Therefore  in  making  a  restriction  of  linearity  on  T,  it  is 
tacitly  assumed  that  the  noise  is  Gaussian  or  nearly  Gaussian  in  the  sense 
that  the  MF  performs  reasonably  well  and  that  any  loss  of  optimality  is 
compensated  for  by  the  simplicity  and  linearity  of  the  MF. 

Under  the  assumption  of  linearity,  the  test  statistic  F(x)  is  equal  to 
the  output  at  time  N  of  a  linear  filter  with  impulse  response  h.  As  a  con¬ 
venience,  the  pseudo-signal  is  defined  to  be  a  length  N  vector  with  ele¬ 
ments  v. i  =  Ayy+j-i,  the  filter  impulse  response  in  reverse  order.  The  out¬ 
put  SNR  of  the  linear  detector  is  found  from  Eq.  (1)  to  be 


SNR,  = 


_  <u  1  s>* 


<u|Ru> 


where  <  |  >  is  standard  inner  product  notation,  R  is  the  noise  covariance 


matrix,  and  A  is  the  maximum  value  of  SNR  for  the  optimal  pseudo-signal. 
Cross-multiplication  yields 


u)  =  <u|s>8  -  A  <u|Ru>  «  0 

This  can  be  maximized  in  the  usual  way  by  setting  its  gradient  equal  to 
zero: 


VL(u)  -  2<u|s>s  -  2ARu  =  0 

Rearranging  and  noting  that  A/  <u|  s>  is  a  constant  and  can  be  set  equal 
to  unity  with  no  loss  of  generality,  the  result  is  the  well  known  MF  equa¬ 
tion: 


s  =  (A/  <u|  s>)Ru  =  Ru  (2) 

The  solution  of  Eq.  (2)  is  the  pseudo-signal  of  the  MF: 

u  =  R_1* 

with  output  SNR  given  by 
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SNR„  =  <u|a>  =  <s|R-1s> 

In  discrete  time  and  with  a  fixed  signal,  the  MF  matrix  equation  can 
be  solved  quite  efficiently  using  the  Levinson  algorithm.  In  continuous 
time,  the  classical  method  of  solution  is  to  use  spectral  factorization  to 
solve  the  equation  on  an  infinite  interval;  this  (possibly)  non-causal  solu¬ 
tion  is  then  projected  onto  a  causal  space  [3].  In  discrete  time,  there  is  a 
parallel  spectral  approach  using  the  eigenvectors  and  values  of  R 

SIGNAL  SELECTION  AND  BOUNDS  ON  THE  SNR 

It  is  well  known  that  the  MF  is  the  linear  filter  with  the  maximum 
SNR0  for  a  given  signal  in  noise.  In  addition  for  non-white  noise,  the  SNRo 
of  the  MF  can  be  maximized  by  proper  choice  of  signal  shape.  Because  of 
this,  for  signals  of  constant  energy,  the  SNRo  of  the  MF  has  a  range  of 
possible  values. 

Since  the  N*N  covariance  matrix  R  is  positive  definite  and  Hermi- 
tian,  it  has  positive,  real  eigenvalues: 

0  <  S  Xg  S  •  '  '  fi  X^y 

and  a  corresponding  set  of  orthonormal  eigenvectors: 

el>  e2 . eN 

The  matrix  R  can  be  diagonalized: 

R=EAE“5 

where  E  is  the  eigenvector  matrix: 

E  =  [elt  ez,  ....  ejsr] 

and  A  is  a  matrix  of  eigenvalues: 

t.  1 


0 

A  = 

0  \N 


Likewise  R"1  has  the  diagonal  form: 


R-1  =  EA^ET1 

where  A-1  has  as  its  diagonal  elements  the  eigenvalues  of  R-1: 

A.*  _U  . . .  *  _ L 

Thus  the  MF  equation  has  the  solution: 

u  =  EA-1E-1s  (3) 

The  signal  s  can  be  expanded  in  terms  of  the  eigenvectors: 

s  —  Ec  (4) 

where  c  is  the  coordinate  of  a  under  the  basis  formed  by  the  orthonormal 

eigenvectors  of  R  From  Eqs.  (3)  and  (4)  the  pseudo-signal  is 

u  =  EA-1c  (5) 

The  SNR,,  of  the  MF  becomes 

SNR,  =  crA-1c 

If  the  signal  is  chosen  to  be  in  the  eigenspace  of  the  ith  eigenvector 
(s  =  kei),  then  the  MF  is  a  simple  correlator  (u  =  s)  and 

SNR,  =  k/\ 

The  Rayleigh  quotient  theorem  [2]  states  that 

J_<  <al  R~*a>  J_ 

I!  S II 2  Ai 

where  the  upper  and  lower  bounds  are  achieved  for  a  signals  in  the  eigen¬ 
space  of  ej  and  e#  respectively.  Thus  the  SNR  of  the  MF  is  bounded: 

SNR,  s;  (0) 

Af/  hi 

The  best  choice  of  signal  is  ej,  the  eigenvector  of  R  with  the  smallest 
eigenvalue.  This  is  equivalent  to  putting  the  signal  is  that  part  of  the 
spectrum  of  R  where  the  noise  has  the  smallest  magnitude. 


Grettenberg  [7]  has  taken  the  logical  step  of  using  M  eigenvectors  as 
an  M  character  alphabet  of  signals.  By  choosing  the  eigenvectors  of  R 
corresponding  to  the  smallest  M  eigenvalues,  not  only  is  the  set  orthogo¬ 
nal,  but  it  achieves  the  greatest  minimum  SNRo  of  any  such  M  character 
set.  This  also  has  an  advantage  of  simplicity,  since,  when  the  signal  is 
chosen  to  be  an  eigenvector  of  R,  from  Eq.  (9)  the  pseudo-signal  equals 
the  signal,  and  the  MF  reduces  to  a  simple  correlator. 

A  minimax  strategy  is  used  by  Turin  [5]  to  find  the  worst-case  noise, 
and  the  corresponding  best  signals  in  continuous  time.  He  shows  that  the 
best  signal  spectrum  should  consist  of  the  noise  spectral  components 
with  the  smallest  magnitude.  As  a  consequence,  '.he  worst  spectra  and 
the  best  signal  both  have  flat  spectra. 


LEVINSON  ALGORITHM  AND  OPTIMAL  SIGNAL  SELECTION 


For  the  Levinson  algorithm  to  produce  the  s-optimal  MF  on  each 
iteration,  the  length  N  optimal  eigenvector  has  to  be  a  truncated 
version  of  the  length  N+i  eigenvector 


eW 


e<*+1>  = 

Let  R^*)  be  the  (W+l)x(Ar+l)  covariance  matrix  with  elements 
then 


rl»-il 


R(*+i)e(tf+i)  -  x(*+i)  e^+1> 

where  is  the  eigenvector  corresponding  to  the  eigenvector  e^+1^. 

Noting  that  the  NxN  minor  of  the  covariance  matrix  is 


_  x(tf+i)e(iv+i)  _ 


r(A0  rW. 

(tM)t  r0 

*N+l 

(rW)TeW 

+  «*+i 

r° 

where 


rW  =  [r^  rN^  ■  ■  ■  rtf 

For  to  be  a  truncated  version  of  e^+1\  then  for  all  N  1: 

(A<*+1>  _  AW)eW  = 

Solving  this  equation  iteratively  yields  permissible  autocovariance 
sequences.  Let  rp  be  the  first  non2ero  term  in  the  covariance  sequence 
after  r0.  Then  every  pth  term  with  index  less  than  L  is  nonzero,  and  the 
rest  are  zero.  All  nonzero  terms  have  the  same  magnitude  with  alternat¬ 
ing  or  constant  sign.  The  covariance  sequences  have  the  form: 

(^[spn^)]*"1  fc  =  o 

rip*k  -  o  k  -  1,  2 . p-1  or  ip+kitL 

where  i  >  0,  0  si  ^p— 1,  p  >  0,  and  Z,  at  0.  As  a  special  case,  if  L-*  0  or 

p-*m>  the  result  is  white  noise,  r*  =  0  for  ah  i  &  1. 

This  places  a  severe  restriction  on  the  noise  autocovariance 
sequence,  and  places  corresponding  limits  on  the  utility  of  the  Levinson 
algorithm  for  this  particular  problem. 

APPROXIMATE  BOUNDS  ON  THE  SNR 

It  is  impractical  to  find  a  suitable  filter  length  N  from  the  bounds  in 
Eq.  (6)  since  they  require  knowledge  of  the  eigenvalues  of  each  AfxAf 
minor  of  R.  Looser  but  easier  to  compute  bounds  can  be  found. 

The  equivalent  rectangular  time  duration  LT  of  the  noise  autocovari¬ 
ance  is  introduced  as  a  rough  measure  of  correlation  [3]: 

AT=  2 

where  o2  =  r0  is  the  noise  variance.  The  largest  eigenvalue  of  R,  denoted 


by  Xjy  is  well  known  to  be  the  smallest  norm  of  R  thus  using  another 


norm: 


\n  *  I|R|I-  -  max£r-M  *  <?LT 

*  i 

This  yields  the  looser  bound: 

MLS  jai_sJMLsSNR„ 

c fAT  ma  x£r^  XN 
1  i 

An  upper  bound  can  be  found.  The  condition  number  of  a  matrix  is 
defined  as: 


K  =  A™./  X—  =  IIR-1  II  IIR'! 


'max'  fXmm 


then  from  Eq.  (6) 


SNR*  ||s|| ZK/Xmax 
The  trace  of  R  equals  the  sum  of  its  eigenvalues: 


therefore 


and  so 


<r(R)  = 

<■1 


o2  ^  *  No 2 


SNRo  s  — 


Since  the  input  SW/?  is  given  by 


snr(  =  M^- 

the  improvement  in  SNR  of  the  MF  is  given  by 
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EXAUPLES 

As  a  first  example,  consider  white  noise  with  an  NxN  covariance 
matrix  R  =  a2 1.  The  covariance  matrix  has  an  Nth  order  eigenvalue  o 2 
making  the  upper  and  lower  bounds  in  Eq.  (6)  equal,  and  the  choice  of  sig¬ 
nal  arbitrary.  Other  considerations,  such  as  a  ceiling  on  transmitted  sig¬ 
nal  strength,  may  still  make  the  spreading  of  signal  energy  in  time  desir¬ 
able. 

For  N  =  3,  the  autocorrelation  sequence  is  given  by 

r  =  [l  r,  rz]T 

For  R  to  be  positive  definite,  the  values  that  rx  and  r2  can  take  are  res¬ 
tricted: 

kj|,|r2j<l  and  |r, |  <  V(r2+l)/2 
This  region  of  the  r,xr2  plane  is  shown  in  Figs.  1  and  2. 

The  difference  in  dB  between  the  upper  and  lower  SNR  bounds  in  Eq. 
(6)  is  plotted  as  contours  in  Fig.  1.  Even  for  a  filter  this  short,  the  signal 
selection  is  shown  to  be  quite  important. 

In  Fig.  2,  the  SNR  of  the  MF  for  an  alternating  signal  (s*  =  (-1)*)  is 
shown  in  dB  over  the  lower  bound.  The  alternating  signal  was  chosen  as  a 
suboptimal  approximation  to  the  optimal  signal  because  of  its  simplicity, 
and  similarity  in  shape  to  the  optimal  signal  for  rj  i  0.  It  is  readily  seen 
to  be  nearly  optimal  in  this  case.  Because  of  the  symmetry  of  this  prob¬ 
lem,  a  constant  signal  (s,  =  1),  chosen  as  a  suboptimal  signal  for  r*  <  0 
has  performance  contours  which  are  the  mirror  image  of  those  in  Fig.  2. 

Four  noise  autocorrelation  functions  were  chosen  as  representative  — 


the  exponential: 


'I 

Fig.  2.  Contours  oT  SNR*,,  for  alternating 
signal  in  r,xrj  plane. 
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the  triangular: 


the  Gaussian: 


ri  ~  exp(-0.2  |i  |) 


l-KI/10  |i|slO 
0  lit  >  10 


rt  -  exp(-n(i/  10)2) 


and  the  hyperbolic  secant: 


rt  =  sech(rri/  10) 

The  exponential  is  the  simplest  member  of  the  Markov  class;  the  triangu¬ 
lar  function  has  finite  supoort;  the  Gaussian  correlation  function  has 
infinite  support,  yet  has  tails  which  fall  off  faster  than  the  exponential, 
and  the  hyperbolic  secant  has  a  nearly  Gaussian  shape  at  the  origin,  but 
exponential  tails. 

The  upper  and  lower  bounds  on  the  SNRj^  from  Eq.  (6)  are  plotted  in 
dB  versus  filter  length  N  in  Figs.  3-6.  Here  signal  selection  is  extremely 
important  for  all  N  >  2  and  increasingly  so  for  increasing  N.  Even  for  the 
length  5  filter,  the  difference  between  the  best  and  worse-case  SNRjijp  is 
at  least  15dB  for  all  four  cases.  At  N  =  20,  the  difference  is  at  least  19dB. 

The  parameters  of  these  four  correlation  functions  were  chosen  so 
that  each  has  an  equivalent  rectangular  time  duration  of  A T  «  10.  Thus 
the  approximate  lower  bound  (Eq.  8)  for  each  function  is 

N /  10  s  SNR up 

The  approximate  and  exact  lower  bounds  for  each  correlation  function  is 
shown  is  Figs.  7-10. 
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Fig.  3.  Upper  and  lower  bounds  on  SS'R^r  for  exponential  correlation. 
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filter  length.  N 

Fig.  5.  Upper  and  lower  bounds  on  S\Ryf  for  Gaussian  correlation. 
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Fig.  7.  Approximate  and  exact  lower  bound 
for  exponentia1  correlation. 


Fig.  8.  Approximate  and  exact  lower  bound 
for  triangular  correlation. 


filter  length.  A' 

Fig.  9.  Approximate  and  exact  lower  bound 
for  Gaussian  correlation. 


filter  length.  A’ 


Fig.  10.  Approximate  and  exact  lower  bound 
for  hyperbolic  secant  correlation. 


CONCLUSIONS 


The  bounds  on  SNR^  in  Eq.  (6)  show  the  selection  of  the  signal  to  be 
important  for  optimal  performance  of  the  MF.  The  selection  of  a  subop- 
timal  signal,  if  made  intelligently,  can  produce  nearly  optimal  results, 
and  certainly  the  importance  of  signal  shape  should  not  be  overlooked. 

The  approximate  lower  bound  of  Eq.  (8)  gives  a  simple  although  con¬ 
servative  estimate  of  worse  case  MF  performance.  An  estimate  of  filter 
length  can  be  made  with  only  knowledge  of  the  equivalent  rectangular 


time  duration  AT. 
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CHAPTER  8  -  CONCLUSIONS 


Review  and  Ideas  for  Further  Research 

In  this  chapter,  some  of  the  main  points  covered  in  the  previous 
chapters  are  reviewed  briefly,  and  some  ideas  for  further  research  are 
given. 

In  Chapter  2,  the  performance  of  three  commonly  used  detectors  is 
considered  for  several  families  of  noise  densities  as  the  input  signal-to- 
noise  ratio  (SNR)  varies.  The  three  detectors  investigated  are  the  linear 
detector,  the  sign  detector  and  the  amplifier  limiter  detector.  Perhaps 
the  most  interesting  results  of  this  chapter  are  the  contours  of  equal 
detector  performance  which  illustrate  the  regions  of  relative  superiority 
for  each  detector.  While  the  nonlinearities  chosen  for  analysis  are 
interesting,  other  nonlinearities  could  be  included.  In  particular,  the 
performance  of  the  locally  optimal  detector  for  each  specific  density 
would  be  informative,  since  it  would  give  some  feel  for  the  degradation  in 
performance  which  can  be  expected  when  the  input  SNR  is  greater  than 
zero. 

In  Chapter  3,  suboptimal  detector  nonlinearities  are  investigated. 
The  design  method  consists  of  choosing  a  suboptimal  nonlinearity  and 
finding  the  family  of  densities  for  which  the  nonlinearity  is  optimal.  A 
member  of  this  family  is  then  fitted  to  the  observed  noise,  and  the 
corresponding  detector  is  used.  The  advantages  of  this  approach  are 
threefold.  When  the  optimal  nonlinearity  is  too  complex  to  use,  a 
simpler,  more  tractable  nonlinearity  can  be  chosen.  Because  the  non¬ 
linearity  is  in  effect  fitted  to  an  observed  noise  using  parameter 
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estimates,  the  noise  statistics  need  not  be  known  exactly  and  can  be 
nonstationary  As  a  result,  this  procedure  can  produce  simple  adaptive 
detectors.  In  Chapter  3,  the  parameters  to  be  estimated  are  assumed 

« 

known  exactly  before  the  analysis  of  detector  performance  is  made.  One 
reasonable  extension  of  this  work  is  to  study  the  sensitivity  of  the  detec¬ 
tor  performance  to  variations  normally  experienced  in  parameter  esti¬ 
mates.  A  more  difficult,  but  possibly  quite  fruitful  area  for  continued 
study,  is  the  problem  of  how  best  to  choose  from  among  suboptimal  non- 
linearities,  or  equivalently,  how  to  choose  a  good  noise  model  when  the 
actual  noise  statistics  are  incompletely  known. 

The  moment  analysis  of  Arctic  under-ice  ambient  noise  is  included  in 
Chapter  4  to  give  an  example  of  some  of  the  techniques  presented  in 
Chapter  3.  The  results  in  this  chapter  suggest  that  the  noise  is  nonsta¬ 
tionary  and  largely  Gaussian  or  nearly  Gaussian  with  sporadic  bursts  of 
non-Gaussian  noise.  Unfortunately,  insufficient  data  is  available  for  more 
extensive  analysis.  With  a  larger  data  set,  a  more  complete  analysis 
could  be  done;  in  particular,  the  non-Gaussian  bursts  merit  more  atten-  ’ 
tion.  Also,  the  assumptions  of  stationarity  and  independence  and  their 
effects  on  moment  estimators  should  be  investigated  further.  Some 
investigation  of  the  degree  of  nonstationarity  would  be  of  interest,  since 
this  would  relate  directly  to  the  performance  of  the  adaptive  detectors 
proposed  in  Chapter  3. 


In  Chapters  5  and  6,  it  is  assumed  that  a  noise  process  is  both 
significantly  dependent  and  non-Gaussian,  and  that  detector  performance 
would  suffer  considerably  if  either  Gaussian  or  independence  assump- 


tions  were  made.  In  this  case  it  is  seen  that  the  real  problem  becomes 
one  of  choosing  a  reasonable  noise  model.  Several  multivariate  noise 
models  are  considered.  The  value  of  each  is  assessed  not  primarily  by 
the  "goodness  of  fit"  of  a  particular  model,  but  by  the  tractability  of  the 
resulting  detector.  This  may  not  be  too  bad  an  assumption;  most  of  the 
models  considered  probably  offer  a  reasonable  fit.  For  example,  the 
transformation  model  is  a  perfect  fit  up  to  second  order  moments.  How¬ 
ever,  some  investigation  of  the  relative  value  of  each  model  in  fitting  a 
specific  observed  noise  would  also  be  important.  A  number  of  appealing 
detector  structures  are  suggested  in  these  chapters,  but  very  little 
analysis  of  their  performance  is  done.  Some  of  these  detectors,  or  slight 
modifications  of  these  detectors,  might  well  be  simple  but  effective  for 
environments  which  are  both  non-Gaussian  and  dependent. 

In  Chapter  7,  two  topics  related  to  discrete  time  matched  filters  are 
considered,  bounds  on  the  SNR  (used  as  a  measure  of  filter  performance), 
and  signal  selection.  Exact  upper  and  lower  bounds  on  the  SNR  are  given, 
and  a  looser,  but  easier  to  compute,  lower  bound  is  given.  As  a  possible 
extension  of  this  work,  an  approximate  upper  bound  which  is  easier  to 
compute  than  the  exact  bound  could  be  of  value.  In  this  chapter  it  is 
shown  that  the  selection  of  a  suboptimal  signal,  if  made  intelligently,  can 
produce  nearly  optimal  results.  The  signals  used  in  the  examples  of 
Chapter  7  were  chosen  because  of  their  intuitive  appeal.  It  may  be  possi¬ 
ble  to  find  an  orderly  procedure  for  selecting  a  suboptimal  signal. 
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